Characteristic function

Contents

In probability theory, the characteristic function of a random variable... The method of characteristic functions is one of the main tools of the analytic theory of probability. This appears very clearly in the proofs of limit theorems and, in particular, in the proof of the central limit theorem, which generalizes the De Moivre-Laplace theorem.

Definition
If

X

is a scalar random variable with distribution

F_{X}

then its characteristic function is

\begin{equation*}\varphi_{X}(t) = \mathbb{E} e^{i t X} = \int_{\mathbb{R}} e^{i t x} \text{d} F_{X}(x),\end{equation*}

where

i

is the imaginary unit and

t \in \mathbb{R}

.
If

F_{X}(x)

has a density

f=f(x)

then

\begin{equation*}\varphi_{X}(t) = \mathbb{E} e^{i t X} = \int_{\mathbb{R}} e^{i t x} f(x) \text{d} x .\end{equation*}

In other words, in this case the characteristic function is just the Fourier transform of

f(x)

Examples

Before dive into the theory of let's consider some examples of characteristic functions.

Coin flip

P(X = 1) = P(X = -1) = 1/2

, then

\begin{equation*}\mathbb{E} e^{itX} = \frac{(e^{it} + e^{-it})}{2}\end{equation*}

Poisson distribution

P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}

for

k = 0, 1, 2, \ldots

, then

\begin{equation*}\mathbb{E} e^{itX} = \sum_{k=0}^{\infty} \frac{e^{-\lambda} \lambda^k e^{ik t}}{k!} = \exp(\lambda(e^{it} - 1))\end{equation*}

Normal distribution

X \sim \mathcal{N}(\mu, \sigma^2)

with probability density function

\begin{equation*}\frac{1}{\sqrt{2\pi \sigma}} \exp(- \frac{(x - \mu)^2}{2 \sigma^2})\end{equation*}

then the characteristic function will be

\begin{equation*}\exp( i \mu t - \frac{\sigma^2 t^2}{2}).\end{equation*}

Properties of characteristic function

Let

X

be a random variable with distribution function

F=F(x)

and

\varphi(t)=\mathrm{E} e^{i t X}

its characteristic function. Then

\varphi

has the following properties:

A. $\varphi(0) = 1$

This property is obvious.

B. $\varphi( - t) = \overline{\varphi(t)}$

\begin{equation*}\varphi(-t) = \mathbb{E} [ \cos(-t X) + i \sin(-t X)] = \mathbb{E} [ \cos(t X) - i \sin(t X)) ]\end{equation*}

C. $| \varphi(t) | \leq 1$

By Jensen inequality and the fact that

| \cdot |

is convex,

\begin{equation*}| \varphi(t) | = | \mathbb{E} e^{itX} | \leq \mathbb{E} | e^{itX} | \leq 1\end{equation*}

D. $\varphi(t)$ is uniformly continuous on $\mathbb{R}$

Again, by Jensen inequality and the fact that

| \cdot |

is convex,

\begin{align*}|\varphi(t + h) - \varphi(t)| &= \left| \mathbb{E} \left(e^{i(t+h)X} - e^{itX}\right) \right| \leq \\&\leq \mathbb{E} \left| e^{i(t+h)X} - e^{itX} \right| = \mathbb{E} \left| e^{itX} ( e^{ihX} - 1) \right| \leq \\&\leq \mathbb{E} \left| e^{ihX} - 1) \right|,\end{align*}

, so uniform convergence follows from the Dominated convergence theorem and the fact the

| e^{ihX} - 1 | \leq 2

a.e.

D. $\mathbb{E} e^{ i t (aX + b)} = e^{ i t b} \varphi(a t)$

The proof is obvious.

Uniqueness

The following theorem shows that the characteristic function is uniquely determined by the distribution function.

Theorem (Uniqueness)
Let

F

and

G

be distribution functions with the same characteristic function, i.e.

\begin{equation*}\int_{-\infty}^{\infty} e^{i t x} d F(x)=\int_{-\infty}^{\infty} e^{i t x} d G(x)\end{equation*}

for all

t \in R

. Then

F(x) \equiv G(x)

Proof. Choose

a

and

b \in R

, and

\varepsilon>0

, and consider the function

f^{\varepsilon}=f^{\varepsilon}(x)

shown in Figure 33. We show that

\begin{equation*}\int_{-\infty}^{\infty} f^{\varepsilon}(x) d F(x)=\int_{-\infty}^{\infty} f^{\varepsilon}(x) d G(x) .\end{equation*}

Let

n \geq 0

be large enough so that

[a-\varepsilon, b+\varepsilon] \subseteq[-n, n]

, and let the sequence

\left\{\delta_n\right\}

be such that

1 \geq \delta_n \downarrow 0, n \rightarrow \infty

. Like every continuous function on

[-n, n]

that has equal values at the end points,

f^{\varepsilon}=f^{\varepsilon}(x)

can be uniformly approximated by trigonometric polynomials (Weierstrass's theorem), i.e. there is a finite sum

\begin{equation*}f_n^{\varepsilon}(x)=\sum_k a_k \exp \left(i \pi x \frac{k}{n}\right)\end{equation*}

such that

\begin{equation*}\sup _{-n \leq x \leq n}\left|f^{\varepsilon}(x)-f_n^{\varepsilon}(x)\right| \leq \delta_n .\end{equation*}

Let us extend the periodic function

f_n(x)

to all of

R

, and observe that

\begin{equation*}\sup_{x} \left|f_n^{\varepsilon}(x)\right| \leq 2\end{equation*}

Then, since by (9)

\begin{equation*}\int_{-\infty}^{\infty} f_n^{\varepsilon}(x) d F(x)=\int_{-\infty}^{\infty} f_n^{\varepsilon}(x) d G(x),\end{equation*}

we have

\begin{align*}\left|\int_{-\infty}^{\infty} f^{\varepsilon}(x) d F(x)-\int_{-\infty}^{\infty} f^{\varepsilon}(x) d G(x)\right| & =\left|\int_{-n}^n f^{\varepsilon} d F-\int_{-n}^n f^{\varepsilon} d G\right| \\\leq & \left|\int_{-n}^n f_n^{\varepsilon} d F-\int_{-n}^n f_n^{\varepsilon} d G\right|+2 \delta_n \\\leq & \left|\int_{-\infty}^{\infty} f_n^{\varepsilon} d F-\int_{-\infty}^{\infty} f_n^{\varepsilon} d G\right|+2 \delta_n \\& +2 F(\overline{[-n, n]})+2 G(\overline{[-n, n]})\end{align*}

where

F(A)=\int_A d F(x), G(A)=\int_A d G(x)

. As

n \rightarrow \infty

, the right-hand side of (13) tends to zero, and this establishes (10).As

\varepsilon \rightarrow 0

, we have

f^{\varepsilon}(x) \rightarrow I_{(a, b]}(x)

. It follows from (10) by the theorem on distribution functions' being the same.

\begin{equation*}\int_{-\infty}^{\infty} I_{(a, b]}(x) d F(x)=\int_{-\infty}^{\infty} I_{(a, b]}(x) d G(x)\end{equation*}

i.e.

F(b)-F(a)=G(b)-G(a)

. Since

a

and

b

are arbitrary, it follows that

F(x)=G(x)

for all

x \in R

.This completes the proof of the theorem.

The inversion formula for characteristic function

The next theorem gives an explicit representation of distribution function

F

in terms of characteristic function

\varphi

, such that

\begin{equation*}\varphi(t)=\int_{-\infty}^{\infty} e^{i t x} \text{d} F(x).\end{equation*}

General case

For pairs of points

a

and

b(a<b)

at which

F=F(x)

is continuous,

\begin{equation*}F(b)-F(a)=\lim _{c \rightarrow \infty} \frac{1}{2 \pi} \int_{-c}^c \frac{e^{-i t a}-e^{-i t b}}{i t} \varphi(t) d t ;\end{equation*}

\quad

Proof. Let's consider the integral

\begin{align*}I_T &=\frac{1}{2 \pi} \int_{-T}^T \frac{e^{-i t a}-e^{-i t b}}{i t} \varphi(t) \text{d} t = \\&= \frac{1}{2 \pi} \int_{-T}^T \int_{\mathbb{R}} \frac{e^{-i t a}-e^{-i t b}}{i t} e^{i t x} \text{d} F(x) \text{d} t\end{align*}

The integrand may look bad near

t = 0

but if we observe that

\begin{equation*}\left| \frac{e^{-ita} - e^{-itb}}{it} \right| = \left| \frac{e^{-ita} - e^{-itb}}{it} \right| = \left| \int_a^b e^{-itx} \, dx \right| \leq b - a\end{equation*}

and

\begin{equation*}\int_{-T}^{T} \int_{-\infty}^{\infty} (b - a) \, dF(x) \, dt \leq 2T(b - a) < \infty.\end{equation*}

Now we can apply Fubini theorem and the fact that

\cos (x)

is even function

\begin{align*}I_T &= \frac{1}{2 \pi} \int_{\mathbb{R}} \int_{-T}^T \frac{e^{-i t a}-e^{-i t b}}{i t} e^{i t x} \text{d} t \text{d} F(x) = \\&= \int_{\mathbb{R}} \left\{ \int_{-T}^{T} \frac{\sin(t(x - a))}{t} \, \text{d} t - \int_{-T}^{T} \frac{\sin(t(x - b))}{t} \, \text{d} t \right\} \text{d} F(x) = \\&= \int_{\mathbb{R}} \left\{ \int_{-T(x - a)}^{T(x - a)} \frac{\sin(u)}{u} \, \text{d} u - \int_{-T(x - b)}^{T(x - b)} \frac{\sin(v)}{v} \, \text{d} v \right\} \text{d} F(x)\end{align*}

Define the integrand function as

\begin{equation*}\Psi_T(x) = \int_{-T(x - a)}^{T(x - a)} \frac{\sin(u)}{u} \, \text{d} u - \int_{-T(x - b)}^{T(x - b)} \frac{\sin(v)}{v} \, \text{d} v\end{equation*}

The function

\begin{equation*}g(s, t)=\int_s^t \frac{\sin v}{v} d v\end{equation*}

is uniformly continuous in

s

and

t

, and

\begin{equation*}g(s, t) \rightarrow \pi\end{equation*}

s \downarrow-\infty

and

t \uparrow \infty

.
Hence there is a constant

C

such that

\left|\Psi_T(x)\right|<C<\infty

for all

T

and

x

. Moreover, it follows that

\begin{equation*}\Psi_T(x) \rightarrow \Psi(x), \quad T \rightarrow \infty,\end{equation*}

where

\begin{equation*}\Psi(x)= \begin{cases}0, & x<a, x>b, \\ \frac{1}{2}, & x=a \; \text{or} \: x=b, \\ 1, & a<x<b .\end{cases}\end{equation*}

Let

\mu

be a measure on

(\mathbb{R}, \mathcal{B}(\mathbb{R}))

such that

\mu(a, b]=F(b)-F(a)

. Then if we apply the dominated convergence theorem, we find that, as

T \rightarrow \infty

\begin{align*}I_T & =\int_{-\infty}^{\infty} \Psi_T(x) d F(x) \rightarrow \int_{-\infty}^{\infty} \Psi(x) d F(x) \\& =\mu(a, b)+\frac{1}{2} \mu\{a\}+\frac{1}{2} \mu\{b\} \\& =F(b-)-F(a)+\frac{1}{2}[F(a)-F(a-)+F(b)-F(b-)] \\& =\frac{F(b)+F(b-)}{2}-\frac{F(a)+F(a-)}{2}=F(b)-F(a),\end{align*}

where the last equation holds for all points

a

and

b

of continuity of

F(x)

\Box

When $\varphi(t)$ is intergable

\int_{-\infty}^{\infty}|\varphi(t)| \text{d} t<\infty

, then distribution function

F(x)

has a density

f(x)

given by

\begin{equation*}f(x)=\frac{1}{2 \pi} \int_{-\infty}^{\infty} e^{-i t x} \varphi(t) \text{d} t .\end{equation*}

\quad

Proof. Let

\int_{-\infty}^{\infty}|\varphi(t)| d t<\infty

. Write

\begin{equation*}f(x)=\frac{1}{2 \pi} \int_{-\infty}^{\infty} e^{-i t x} \varphi(t) d t .\end{equation*}

It follows from the dominated convergence theorem that this is a continuous function of

x

and therefore is integrable on

[a, b]

. Consequently we find, applying Fubini's theorem, that

\begin{align*}\int_a^b f(x) d x & =\int_a^b \frac{1}{2 \pi}\left(\int_{-\infty}^{\infty} e^{-i t x} \varphi(t) d t\right) d x \\& =\frac{1}{2 \pi} \int_{-\infty}^{\infty} \varphi(t)\left[\int_a^b e^{-i t x} d x\right] d t=\lim _{c \rightarrow \infty} \frac{1}{2 \pi} \int_{-c}^c \varphi(t)\left[\int_a^b e^{-i t x} d x\right] d t \\& =\lim _{c \rightarrow \infty} \frac{1}{2 \pi} \int_{-c}^c \frac{e^{-i t a}-e^{-i t b}}{i t} \varphi(t) d t=F(b)-F(a)\end{align*}

for all points

a

and

b

of continuity of

F(x)

.Hence it follows that

\begin{equation*}F(x)=\int_{-\infty}^x f(y) d y, \quad x \in R,\end{equation*}

and since

f(x)

is continuous and

F(x)

is nondecreasing,

f(x)

is the density of

F(x)

.This completes the proof of the theorem.

\Box

References

[Durrett2019]
Rick Durrett. Probability Theory and Examples. Fifth edition. 2019.

Characteristic function

Examples

Properties of characteristic function

The inversion formula for characteristic function

Examples

Examples

Coin flip

Poisson distribution

Normal distribution

Properties of characteristic function

Properties of characteristic function

A. φ(0)=1\varphi(0) = 1φ(0)=1

B. φ(−t)=φ(t)‾\varphi( - t) = \overline{\varphi(t)}φ(−t)=φ(t)​

C. ∣φ(t)∣≤1| \varphi(t) | \leq 1∣φ(t)∣≤1

D. φ(t)\varphi(t)φ(t) is uniformly continuous on R\mathbb{R}R

D. Eeit(aX+b)=eitbφ(at)\mathbb{E} e^{ i t (aX + b)} = e^{ i t b} \varphi(a t)Eeit(aX+b)=eitbφ(at)

Uniqueness

Uniqueness

The inversion formula for characteristic function

The inversion formula for characteristic function

General case

When φ(t)\varphi(t)φ(t) is intergable

References

References

A. $\varphi(0) = 1$

B. $\varphi( - t) = \overline{\varphi(t)}$

C. $| \varphi(t) | \leq 1$

D. $\varphi(t)$ is uniformly continuous on $\mathbb{R}$

D. $\mathbb{E} e^{ i t (aX + b)} = e^{ i t b} \varphi(a t)$

When $\varphi(t)$ is intergable