B.2. Slutsky’s Theorem

Theorem 6.B.1 can be used to prove Theorem 6.7. Theorem 6.3 was only proved for the special case that the probability limit Xis constant. However, the general result of Theorem 6.3 follows straightforwardly from Theorems 6.7 and 6.B.3. Let us restate Theorems 6.3 and 6.7 together:

Theorem 6.B.4: (Slutsky’s theorem). Let Xn a sequence of random vectors in Kk converging a. s. (inprobability) to a (random or constant) vectorX. Let Ф(x) be an Rm-valued function on Kk that is continuous on an open (Borel) set B in Rk for which P(X є B) = 1). Then Ф(Xn) converges a. s. (inprobability) to Ф( X).

Proof: Let Xn ^ X a. s. and let {^, tX, P} be the probability space in­volved. According to Theorem 6.B.1 there exists a null set N1 such that lim„^Z Xn(ш) = X(of pointwise in ш є N1. Moreover, let N2 = {ш є ^ : X(ш) / B}. Then N2 is also a null set and so is N = N1 U N2 . Pick an ar­bitrary ш є ЙN. Because Ф is continuous in X(ш) it follows from standard calculus that lim„^Z Ф(Xn(«)) = Ф(X(ш)). By Theorem 6.B.1 this result im­plies that Ф(Xn) ^ ^(X) a. s. Because the latter convergence result holds along any subsequence, it follows from Theorem 6.B.3 that Xn ^p X implies ^(X„) ^p Ф(X). Q. E.D.

6.B.3. Kolmogorov’s Strong Law of Large Numbers

I will now provide the proof of Kolmogorov’s strong law of large numbers based on the elegant and relatively simple approach of Etemadi (1981). This proof (and other versions of the proof as well) employs the notion of equivalent sequences.

Now let Xn, n > 1 be the sequence in Lemma 6.B.2, and suppose that (1/n)£n=1 max(0, Xj) ^ E[max(0, X1)] a. s. and (1/n)J^’j=1max(0, —Xj) ^ E[max(0, — X1)] a. s. Then it is easy to verify from Theorem 6.B.1, by taking the union of the null sets involved, that

For each natural number k > a there exists a natural number nk such that [ank] < k < [ank+1], and since the Xj’s are nonnegative we have

[ank ] [ank +1]

Ї^т+Г]z tt"n* 1) — Z <» — Z (Kk+1B • <«.7°)

The left-hand expression in (6.70) converges a. s. to E[X 1]/a as k — x, and the right-hand side converges a. s. to aE[X1]; hence, we have, with probabi­lity 1,

—E [X1] — liminf Z (k) — limsup Z (k) — a E [X1]^

a k—x k——x

In other words, if we let Z = liminfk—x Z (k), Z = limsupk—x Z (k), there ex­ists a null set Na (depending on a) such that for all ш є Na, E[X1]/a — Z (ш) — Z(rn) — a E [X1]. Taking the union N of Na over all rational a > 1, so that N is also a null set,8 we find that the same holds for all ш є ^N and all rational a > 1. Letting a I 1 along the rational values then yields limk—x Z(k) = Z(ш) = 2(ш) = E[X1] for all ш є ^N. Therefore, by The­orem 6.B.1, (1/n)Y^j=1Yj — E[X1] a. s., which by Lemmas 6.B.2 and 6.B.3 implies that (1/n)J^=1 Xj — E[X1]. a. s. Q. E.D.

This completes the proof of Theorem 6.6.

6.B.5. The Uniform Strong Law of Large Numbers and Its Applications Proof of Theorem 6.13: It follows from (6.63), (6.64), and Theorem 6.6 that

However, this set is an uncountable intersection of sets in & and therefore not necessarily a set in & itself. The following lemma, which is due to Jennrich (1969), shows that in the case under review there is no problem.

Proof of Theorem 6.14: Let {Й, P } be the probability space involved, and denote вп = в. Now (6.9) becomes Q(en) — Q(e0) a. s., that is, there exists a null set N such that for all ш e N,

lim Q(0n(ш)) = й(во)- (6.73)

n — ТО

Suppose that for some ш e ^N there exists a subsequence nm (ш) and an є > 0 such that infm>i \вПт(ш)(ш) — в0| > є. Then by the uniqueness condi­tion there exists a 8(ш) > 0 such that Q(00) – <2(впт(ш)(ш)) > 8(ш) for all m > 1, which contradicts (6.73). Hence, for every subsequence nm (ш) we have limm^TO Onm(ш)(ш) = во, which implies that limn—^(ш) = во.

Proof of Theorem 6.15: The condition Xn — c a. s. translates as follows: There exists a null set N1 such that for all ш e ^N1, limn—TO Xn(ш) = c. By the continuity of Ф on B the latter implies that limn—TO |Ф(Х„ (ш)) — Ф(с)| = 0and that for at most a finite number of indices n, Xn (ш) Є B. Similarly, the uniform a. s. convergence condition involved translates as follows: There exists a null set N2 such that for all ш e ^N2, limn—TOsupxeB|Фп(x, ш) — Ф(x)| — 0. Take N = N1 U N2. Then for all ш e QN,