Skip to content

Kolmogorov's Three-Series Theorem

The Kolmogorov's Three Series Theorem
The Kolmogorov's Three Series Theorem is a fundamental result in probability theory that provides conditions under which the sum of an infinite series of independent random variables converges almost surely.
Suppose that X1,X2,…X_1, X_2, \ldots is a sequence of independent random variables, Sn=X1+…+XnS_n = X_1 + \ldots + X_n, and let AA be the set of sample points ω\omega for which ∑i>0Xi(ω)\sum_{i > 0} X_i(\omega) converges to a finite limit. It follows from Kolmogorov's zero-one law that P(A)=0\mathbb{P}(A) = 0 or 1, i.e. the series ∑i>0Xi(ω)\sum_{i > 0} X_i(\omega) converges or diverges almost surely (a.s.). The aim of this article is to give criteria that will determine whether a sum of independent random variables converges or diverges.

The foolowing theorem is Theorem 1, page 6 [Shiryaev]

This result is due to Kolmogorov and Khinchin.
KOLMOGOROV-KHINCHIN CONVERGENCE THEOREM
Suppose that X1,X2,…X_1, X_2, \ldots is a sequence of independent random variables and EXn=0\mathbb{E} X_n=0. We get
∑nEXn2<∞⟹∑n=1∞Xn converges a.s..\begin{equation*}\sum_n\mathbb{E} X_n^2<\infty \quad \Longrightarrow \quad \sum^{\infty}_{n=1} X_n \text{ converges a.s.}.\end{equation*}
Moreover, if the random variables {Xn,n≥1} \{ X_n, n \geq 1 \}, are uniformly bounded for some cc the converse is true:
 if ∃c<∞ such that P(∣Xn∣≤c)=1 and ∑n=1∞Xn converges a.s.⟹∑nEXn2<∞.\begin{equation*}\text{ if } \exists c < \infty \text{ such that } \mathbb{P}\left(\left|X_n\right| \leq c \right) =1 \text{ and } \sum^{\infty}_{n=1} X_n \text{ converges a.s.} \quad \Longrightarrow \quad \sum_n\mathbb{E} X_n^2<\infty.\end{equation*}
The sequence {Sn,n≥1}\{ S_n , n \geq 1 \}, converges a.s., if and only if it is fundamental a.s.. The sequence {Sn,n≥1}\{ S_n , n \geq 1 \}, is fundamental a.s. if and only if
P{sup⁡k≥1∣Sn+k−Sn∣≥ε}→0,n→∞.\begin{equation}\mathbb{P}\left\{\sup _{k \geq 1}\left|S_{n+k}-S_n\right| \geq \varepsilon\right\} \rightarrow 0, \quad n \rightarrow \infty .\end{equation}P{sup⁡k≥1∣Sn+k−Sn∣≥ε}=lim⁡N→∞P{max⁡1≤k≤N∣Sn+k−Sn∣≥ε}≤Kolmogorovlim⁡N→∞∑k=nn+NEXk2ε2=∑k=n∞EXk2ε2.\begin{equation*}\mathbb{P}\left\{\sup _{k \geq 1}\left|S_{n+k}-S_n\right| \geq \varepsilon\right\} =\lim _{N \rightarrow \infty} \mathbb{P}\left\{\max _{1 \leq k \leq N}\left|S_{n+k}-S_n\right| \geq \varepsilon\right\} \overset{\text{Kolmogorov}}{\leq} \lim _{N \rightarrow \infty} \frac{\sum_{k=n}^{n+N} \mathbb{E} X_k^2}{\varepsilon^2}=\frac{\sum_{k=n}^{\infty} \mathbb{E} X_k^2}{\varepsilon^2} .\end{equation*}
Therefore
∑k=1∞EXk2<∞⟹Sn is Cauchy sequence a.s. \begin{equation*}\sum_{k=1}^{\infty} \mathbb{E} X_k^2<\infty \quad \Longrightarrow \quad S_n \text{ is Cauchy sequence a.s. }\end{equation*}
and hence lim⁡n→∞Sn\lim_{n \rightarrow \infty} S_n exists a.s.
Now, let Sn=∑k=1nXkS_n = \sum_{k=1}^{n} X_k converges, then ∃n0\exists n_0
Sn is Cauchy sequence a.s. ⟹P{sup⁡k≥1∣Sn0+k−Sn0∣≥ε}<12.\begin{equation*}S_n \text{ is Cauchy sequence a.s. } \quad \Longrightarrow \quad \mathbb{P}\left\{\sup _{k \geq 1}\left|S_{n_0+k}-S_{n_0}\right| \geq \varepsilon\right\}<\frac{1}{2} .\end{equation*}
By the second part of Kolmogorov's inequality,
P{sup⁡k≥1∣Sn+k−Sn∣≥ε}≥1−(c+ε)2∑k=n∞EXk2.\begin{equation*}\mathbb{P}\left\{\sup _{k \geq 1}\left|S_{n+k}-S_n\right| \geq \varepsilon\right\} \geq 1-\frac{(c+\varepsilon)^2}{\sum_{k=n}^{\infty} \mathbb{E} X_k^2} .\end{equation*}
Therefore if we suppose that ∑k=1∞EXk2=∞\sum_{k=1}^{\infty} \mathbb{E} X_k^2=\infty, we obtain
P{sup⁡k≥1∣Sn+k−Sn∣≥ε}=1,\begin{equation*}\mathbb{P}\left\{\sup _{k \geq 1}\left|S_{n+k}-S_n\right| \geq \varepsilon\right\}=1,\end{equation*}
which contradicts the convergence assumption. □\Box

The foolowing theorem is Theorem 3, page 9 [Shiryaev]

Let cc be a nonegative constant and define the truncated random variables
Xn′={Xn,∣Xn∣≤c,0,∣Xn∣>c.\begin{equation*}X_n^{\prime}= \begin{cases}X_n, & |X_n| \leq c, \\ 0, & |X_n|>c .\end{cases}\end{equation*}
KOLMOGOROV'S THREE-SERIES THEOREM
Let X1,X2,…X_1, X_2, \ldots be a sequence of independent random variables. A necessary and sufficient condition for the convergence of ∑Xn\sum X_n a.s. is that the threes series converge for some c>0c>0
∑n=1∞Xn converges a.s..  ⟺  ∑EXn′,∑Var⁡Xn′,∑P(∣Xn∣≥c).\begin{equation}\sum^{\infty}_{n=1} X_n \text{ converges a.s.}. \iff \sum \mathbb{E} X_n^{\prime}, \quad \sum \operatorname{Var} X_n^{\prime}, \quad \sum \mathbb{P}\left(\left|X_n\right| \geq c\right).\end{equation}
Let's prove this theorem.
To prove it, let μn=EXn′\mu_n = \mathbb{E} X_n^{\prime}. Convergence of ∑Var⁡Xn′\sum \operatorname{Var} X_n^{\prime} and the Kolmogorov-Khinchin convergence theorem imply that ∑n=1∞(Xn′−μn) \sum_{n=1}^{\infty} \left( X_n^{\prime} - \mu_n \right) converges a.s.
∑n=1∞Var⁡Xn′ converges a.s..⟹Kolmogorov-Khinchin∑n=1∞(Xn′−μn) converges a.s..\begin{equation*}\sum^{\infty}_{n=1} \operatorname{Var} X_n^{\prime} \text{ converges a.s.}. \quad \overset{\text{Kolmogorov-Khinchin}}{\Longrightarrow} \quad \sum_{n=1}^{\infty} \left( X_n^{\prime} - \mu_n \right) \text{ converges a.s.}.\end{equation*}
Convergence of ∑μn\sum \mu_n now gives that ∑Xn′\sum X_n^{\prime} converges a.s.
∑n=1∞(Xn′−μn) converges a.s. and ∑n=1∞μn converges a.s..⟹∑n=1∞Xn′ converges a.s..\begin{equation*}\sum_{n=1}^{\infty} \left( X_n^{\prime} - \mu_n \right) \text{ converges a.s.} \quad \text{ and } \quad \sum^{\infty}_{n=1} \mu_n \text{ converges a.s.}. \quad \Longrightarrow \quad \sum_{n=1}^{\infty} X_n^{\prime} \text{ converges a.s.}.\end{equation*}
But if ∑P(∣Xn∣≥c)<∞\sum \mathbb{P}\left(\left|X_n\right| \geq c\right)<\infty, then by the Borel-Cantelli lemma
∑n=1∞P(∣Xn∣≥c)<∞⟹Borel-Cantelli∃M such that ∀n>M,∣Xn∣<c\begin{equation*}\sum^{\infty}_{n=1} \mathbb{P}\left(\left|X_n\right| \geq c\right)<\infty \quad \overset{\text{Borel-Cantelli}}{\Longrightarrow} \quad \exists M \text{ such that } \forall n > M, |X_n| < c\end{equation*}
Therefore Xn=Xn′X_n=X_n^{\prime} for all nn starting from MM. Therefore ∑Xn\sum X_n also converges a.s.
If ∑Xn\sum X_n converges a.s. then Xn→0X_n \rightarrow 0 a.s., and therefore, for every c>0c>0, at most a finite number of the events {∣Xn∣≥c}\left\{\left|X_n\right| \geq c\right\} can occur a.s.. Therefore ∑1{∣Xn∣≥c}<∞\sum \mathbf{1}_{\{|X_n| \geq c\}}<\infty a.s., and, by the second part of the Borel-Cantelli lemma, ∑P(∣Xn∣>c)<∞\sum \mathbb{P}\left(\left|X_n\right|>c\right)<\infty. Moreover, the convergence of ∑Xn\sum X_n implies the convergence of ∑Xn′\sum X_n^{\prime}.
To prove of convergence of the both series ∑EXn′\sum \mathbb{E} X_n^{\prime} and ∑Var⁡Xn′\sum \operatorname{Var} X_n^{\prime} we will use symmetrization method. In addition to the sequence
X1′,X2′,…X_1^{\prime}, X_2^{\prime}, \ldots, we consider a different sequence, X1′^,X2′^,…\widehat{X_1^{\prime}}, \widehat{X_2^{\prime}}, \ldots of independent random variable such Xn′X_n^{\prime} has the same distribution as Xn′^\widehat{X_n^{\prime}}
Xn′∼Xn′^\begin{equation*}X_n^{\prime} \sim \widehat{X_n^{\prime}}\end{equation*}
Then if ∑nXn′\sum_nX_n^{\prime} converges a.s., the series ∑nXn′^\sum_n\widehat{X_n^{\prime}} also converges, and hence so does ∑n(Xn′−Xn′^)\sum_n \left(X_n^{\prime}-\widehat{X_n^{\prime}}\right).
Also we have
Xn′∼Xn′^⟹E(Xn′−Xn′^)=0 and P(∣Xn′−Xn′^∣≤2c)=1\begin{equation*}X_n^{\prime} \sim \widehat{X_n^{\prime}} \quad \Longrightarrow \quad \mathbb{E}\left(X_n^{\prime}-\widehat{X_n^{\prime}}\right)=0 \text{ and } \mathbb{P} \left( \left| X_n^{\prime}-\widehat{X_n^{\prime}} \right| \leq 2 c\right)=1\end{equation*}
Therefore by the first part of Kolmogorov-Khinchin convergence theorem
∑n=1∞(Xn′−Xn′^) and P(∣Xn′−Xn′^∣≤2c)=1⟹Kolmogorov-Khinchin∑n=1∞Var⁡(Xn′−Xn′^)<∞.\begin{equation*}\sum^{\infty}_{n=1} \left(X_n^{\prime}-\widehat{X_n^{\prime}}\right) \text{ and } \mathbb{P} \left( \left| X_n^{\prime}-\widehat{X_n^{\prime}} \right| \leq 2 c\right)=1 \quad \overset{\text{Kolmogorov-Khinchin}}{\Longrightarrow} \quad \sum^{\infty}_{n=1} \operatorname{Var}\left(X_n^{\prime}-\widehat{X_n^{\prime}}\right)<\infty.\end{equation*}
We can get the convergence of ∑nVar⁡Xn′\sum_n\operatorname{Var} X_n^{\prime} and to get the convergence of ∑nEXn′\sum_n\mathbb{E} X_n^{\prime} we use the first part of Kolmogorov-Khinchin convergence theorem,
∑nVar⁡Xn′=12∑nVar⁡(Xn′−Xn′^)<∞⟹Kolmogorov-Khinchin∑(Xn′−EXn′^) converges with a.s..\begin{equation*}\sum_n\operatorname{Var} X_n^{\prime}=\frac{1}{2} \sum_n\operatorname{Var}\left(X_n^{\prime}-\widehat{X_n^{\prime}}\right)<\infty \quad \overset{\text{Kolmogorov-Khinchin}}{\Longrightarrow} \quad \sum\left(X_n^{\prime}-\mathbb{E} \widehat{X_n^{\prime}}\right) \text{ converges with a.s.}.\end{equation*}
and therefore ∑nEXn′\sum_n\mathbb{E} X_n^{\prime} converges.
Thus if ∑nXn′\sum_nX_n^{\prime} converges a.s. (and P(∣Xn′∣≤c)=1,n≥1\mathbb{P}\left(\left|X_n^{\prime}\right| \leq c\right)=1, n \geq 1 ) it follows that both ∑nEXn′\sum_n\mathbb{E} X_n^{\prime} and ∑nVar⁡Xn′\sum_n\operatorname{Var} X_n^{\prime} converge. □\Box

References