The Arzelà–Ascoli Theorem

If $\mathcal F$ is a collection of functions defined on $D \subseteq \mathbb{R}^n$ then $\mathcal F$ is said to be Uniformly Bounded on $D$ if there exists an $M \in \mathbb{R}$, $M > 0$ such that $|f(x)| \leq M$ for all $x \in D$ and for all $f \in \mathcal F$.

Furthermore, $\mathcal F$ is said to be Equicontinuous on $D$ if for all $\epsilon > 0$ there exists a $\delta > 0$ such that if $| x - y | < \delta$ then $| f(x) - f(y) | < \epsilon$ for all $x, y \in D$ and for all $f \in \mathcal F$.

We now state and prove a very important result which states that every sequence of continuous real-valued functions defined on a compact subset $D \subseteq \mathbb{R}^n$ that is uniformly bounded and equicontinuous has a uniformly convergent subsequence to a continuous function on $D$.

In the proof of the Arzelà–Ascoli theorem we use the follow result: If $(f_n(x))_{n=1}^{\infty}$ is a sequence of continuous functions defined on the compact set $D \subseteq \mathbb{R}$ then $(f_n(x))_{n=1}^{\infty}$ is a uniformly Cauchy sequence of functions if and only if there exists a subsequence $(f_{n_k}(x))_{k=1}^{\infty}$ that converges uniformly on $D$ to a function $f \in C(D, \mathbb{R})$.

Since $D \subset \mathbb{R}^n$ is a compact, there exists a countable and dense subset $R = \{ r_1, r_2, ... \}$ of $D$.

Consider the sequence $(f_{0,n}(r_1))_{n=1}^{\infty} = (f_n(r_1))_{n=1}^{\infty}$ of real numbers. Since $\mathcal F$ is uniformly bounded, the sequence $(f_{0,n}(r_1))_{n=1}^{\infty}$ is a bounded sequence of real numbers. By the Bolzano-Weierstrass theorem there exists a convergent subsequence of $(f_{0,n}(r_1))_{n=1}^{\infty}$ which we label $(f_{1,n}(r_1))_{n=1}^{\infty}$ that converges to the number which we label $f(r_1)$.

Now consider the sequence $(f_{1,n}(r_2))_{n=1}^{\infty}$ of real numbers. Once again, since $\mathcal F$ is uniformly bounded, the sequence $(f_{1,n}(r_2))_{n=1}^{\infty}$ is a bounded sequence of real numbers. By the Bolzano-Weierstrass theorem there exists a convergence subsequence of $(f_{1,n}(r_2))_{n=1}^{\infty}$ which we label $(f_{2,n}(r_2))_{n=1}^{\infty}$ that converges to the number which we label $f(r_2)$.

We continue constructing convergent subsequences in this manner.

For each $k \in \mathbb{N}$, we consider the sequence $(f_{k,n}(r_{k+1}))_{n=1}^{\infty}$ of real numbers. Since $\mathcal F$ is uniformly bounded, the sequence $(f_{k,n}(r_{k+1}))_{n=1}^{\infty}$ is a bounded sequence of real numbers. By the Bolzano-Weierstrass theorem there exists a convergent subsequence of $(f_{k,n}(r_{k+1}))_{n=1}^{\infty}$ which we label $(f_{k+1,n}(r_{k+1}))_{n=1}^{\infty}$ that converges to the number which we label $f(r_{k+1})$.

By construction, each row of functions above is a subsequence of all of the rows of functions above it.

Define a new sequence $(g_n(x))_{n=1}^{\infty} = (f_{n,n}(x))_{n=1}^{\infty}$. Then for each $r_j \in R$ we have that the numerical sequence $(g_n(r_j))_{n=1}^{\infty}$ converges to $f(r_j)$.

Now let $\epsilon > 0$ be given. We aim to show that $(g_n(x))_{n=1}^{\infty}$ is a Cauchy sequence of functions.

For each $r_j \in R$, since $(g_n(r_j))_{n=1}^{\infty}$ is a convergent numerical sequence, this numerical sequence is also a Cauchy sequence (since every convergent sequence of real numbers is Cauchy). So for $\displaystyle{\epsilon_1 = \frac{\epsilon}{3}}$ there exists an $M_j \in \mathbb{N}$ such that if $m, n \geq M_j$ then:

Furthermore, since $\mathcal F$ is equicontinuous, for $\displaystyle{\epsilon_2 = \frac{\epsilon}{3} > 0}$ there exists a $\delta^* > 0$ such that for all $x, y \in D$ with $| x - y | < \delta^*$ and for all functions $g_n \in \mathcal F$ we have that:

So if $m, n \geq M$ and $x \in D$ then $x \in B(r_{j_k}, \delta)$ for some $k \in \{ 1, 2, ..., l \}$ and from above, $| g_m(x) - g_n(x) | < \epsilon$. So $(g_n(x))_{n=1}^{\infty}$ is a uniformly Cauchy sequence. But recall that a sequence of functions $(g_n(x))_{n=1}^{\infty}$ defined on a compact set $D$ is a uniformly Cauchy sequence if and only if $(g_n(x))_{n=1}^{\infty}$ converges uniformly to a continuous function $f$ defined on $D$.

Hence $(g_n(x))_{n=1}^{\infty}$ is a subsequence of $(f_n(x))_{n=1}^{\infty}$ that converges uniformly to a function $f \in C(D, \mathbb{R})$. $\blacksquare$