De Moivre–Laplace theorem, which is a special case of the central limit theorem, states that the normal distribution may be used as an approximation to the binomial distribution under certain conditions.
The theorem appeared in the second edition of The Doctrine of Chances by Abraham de Moivre, published in 1738. Although de Moivre did not use the term "Bernoulli trials", he wrote about the probability distribution of the number of times "heads" appears when a coin is tossed 3600 times.
Let X1,X2,… be a sequence of independent and identically distributed random variables, whereX1={1,−1,withP(X1=1)=21,withP(X1=−1)=21.and let Sn=∑k=1nXk. In words, we are betting 1 on the flipping of a fair coin and Sn is our winnings at time n.
Theorem (The De Moivre-Laplace Theorem) The De Moivre-Laplace Theorem. If a<b then as n→∞P(a≤n1/2Sn≤b)→∫ab(2π)−1/2e−z2/2dz
Motivation
Motivation
If n and k are integersP(S2n=2k)=(2nn+k)2−2nsince S2n=2k if and only if there are n+k flips that are +1 and n−k flips that are -1 in the first 2n. The first factor gives the number of such outcomes and the second the probability of each one.
Sum distribution
If we visualize such probabilitites and consequentially increase n we will start see the pattern: the binomial probabilities formed a bell-shaped curve and this curve became smoother as n increased.
Plot of binomial distributions
The transformation from a discrete, jumpy distribution to a smooth, symmetric bell curve represents one of probability theory's most beautiful and fundamental convergence phenomena.
Proof of De Moivre-Laplace Theorem
The foolowing proof is taken from [Durrett2019] p. 98-99
Proof of De Moivre-Laplace Theorem
The foolowing proof is taken from [Durrett2019] p. 98-99
The proof strategy for the theorem is as follows: initially, we approximate the probability P(S2n=2k) for a single point. Subsequently, we proceed to calculate the probability for the interval P(a≤2n1/2S2n≤b).
Approximation of P(S2n=2k)
This subsection provides a proof for the theorem presented below, which approximates the single probability P(S2n=2k).
Theorem If 2k/(2n)1/2→x then P(S2n=2k)∼(πn)−1/2e−z2/2.
By Stirling's formula we haven!∼nne−n(2πn)1/2 as n→∞where an∼bn means an/bn→1 as n→∞. Now we have
(2nn+k)=(n+k)!(n−k)!(2n)!∼(n+k)n+k(n−k)n−k(2n)2n⋅(2π(n+k))1/2(2π(n−k))1/2(2π(2n))1/2and we get(2nn+k)2−2n∼(1+nk)−n−k⋅(1−nk)−n+k⋅(πn)−1/2⋅(1+nk)−1/2⋅(1−nk)−1/2
The first two terms on the right can be transformed by the formula for the difference of two squares(1+nk)−n−k⋅(1−nk)−n+k=(1−n2k2)−n⋅(1+nk)−k⋅(1−nk)k
Using the formula for en→∞lim(1+n1)n=e,we see that if 2k=x(2n)1/2, i.e., k=xn/2, then(1−n2k2)−n=(1−2nx2)−n=(1+2x2⋅−n1)−n→ex2/2(1+nk)−k=(1+(2n)1/2x)−xn/2=(1−2x2⋅−xn/21)−xn/2→e−z2/2(1−nk)k=(1−x/(2n)1/2)xn/2=(1−2x2⋅xn/21)xn/2→e−x2/2
For this choice of k,k/n→0, so(1+nk)−1/2⋅(1−nk)−1/2→1and putting things together gives:P(S2n=2k)∼πn1e−z2/2
Approximation of P(a≤S2n/(2n)1/2≤b)
Our next step is to computeP(a(2n)1/2≤S2n≤b(2n)1/2)=m∈[a(2n)1/2,b2n]∩2Z∑P(S2n=m)Changing variables m=x(2n)1/2, we have that the above isP(a(2n)1/2≤S2n≤b(2n)1/2)≈x∈[a,b]∩(2Z/(2n)1/2)∑(2π)−1/2e−z2/2⋅(2/n)1/2where 2Z/(2n)1/2={2z/(2n)1/2:z∈Z}. We have multiplied and divided by 2 since the space between points in the sum is (2/n)1/2, so if n is large the sum above is≈∫ab(2π)−1/2e−x2/2dxThe integrand is the density of the (standard) normal distribution, so changing notation we can write the last quantity as P(a≤χ≤b) where χ is a random variable with that distribution. We proved the De Moivre-Laplace Theorem. To remove the restriction to even integers observe S2n+1=S2n±1 . □
