Probability space

Sample space

% [1] p. 5Let us consider an experiment of which all possible results are included in a finite number of outcomes

\omega_1, ..., \omega_N

. We do need to know the nature of these outcomes, only that there are a finite number

N

of them. We call

\omega_1, ..., \omega_N

---elementary events---, or ---sample points---, and the finite set

\Omega = \{ \omega_1, ..., \omega_N, \}

the space of elementary events or the sample space. \\The choice of the space of elementary events is the first step in formulating a probabilistic model for an experiment. Let us consider some examples of sample spaces. \\Example (Single coin toss). \\For a single toss of a coin, the sample space n consists of two points

\Omega = \{ H, T \}

where H = "head" and T = "tail". We exclude possibilities like "the coin stands on the edge", "the coin disappears", etc) \\

Example (Multiple coin toss). \\For n tosses of a coin, the sample space is

\Omega = \{ \omega : \omega = (a_1, ..., a_n), a_i = H \quad or \quad T \},

and the total number

N (\Omega)

of outcomes is

2^n

.

Events

In addition to the concept of sample space, we now need the fundamental concept of the ---event---. Experimenters are ordinarily interested, not in what particular outcome occurs as the result of a trial, but in whether the outcome belongs to some subset of the set of all possible outcomes, We shall describe as events all subsets

A \subset \Omega

for which, under the conditions of the experiment, it is possible to say either "the outcome

\omega \in A

" or "the outcome

\omega \notin A

" \\For example, let a coin be tossed three times, The sample space

\Omega

consists of the eight points

\Omega = \{ HHH, HHT, ..., TTT \}

and if we are able to observe (determine, measure, etc) the results of all three tosses, we say that the set

A = \{ HHH, HHT, HTH, THH \}

is the event consisting of the appearance of at least two heads. If, however, we can determine only the results of the first toss, this set A cannot be considered to be an event, since there is no way to give either a positive or negative answer to the question of whether a specific outcome

w

belongs to

A

. \\Starting from a given collection of sets that are events, we can form new events by means of statements containing the logical connectives "or", "and" and "not", which correspond in the language of set theory to the operations "unions", "intersection" and "complement".If

A

and

B

are sets, their ---union---, denoted by

A \cup B

, is the set of points that belong either to

A

or to

B

:

\begin{equation*}A \cup B = \{ \omega \in \Omega : \omega \in A \quad or \quad \omega \in B\}\end{equation*}

In the language of probability theory,

A \cup B

is the event consisting of the realization either of

A

or of

B

.The ---intersection--- of

A

and

B

, denoted by

A \cap B

, is the set of points that belong to both

A

or

B

:

\begin{equation*}A \cap B = \{ \omega \in \Omega : \omega \in A \quad and \quad \omega \in B\}\end{equation*}

The

A \cap B

consists of the simultaneous realization of both

A

and

B

. If

A

is a subset of

\Omega

, its **complement**, denoted by

\hat{A}

, is the set of points of

\Omega

that do not belong to

A

, then

\hat{A} = \Omega \setminus A

. We denote the empty set by

\emptyset

. In probability theory,

\emptyset

is called an impossible event. The set

\Omega

is naturally called the certain event. \\If we consider a collection

\mathcal{A}_0

of sets

A \subset \Omega

we may use the set-theoretic operators

\cup

,

\cap

and

\setminus

to form a new collection of sets from the elements

\mathcal{A}_0

; these sets are again events. If we adjoin the certain and impossible events

\Omega

and

\emptyset

we obtain a collection

\mathcal{A}

of sets which is an algebra, i.e. a collection of subsets of

\Omega

for which

$\Omega \in \mathcal{A}$
if $A \in \mathcal{A}$ , $B \in \mathcal{A}$ , the sets $A \cup B$ , $A \cap B$ , $A \setminus B$ also belong to $\mathcal{A}$

Probability

We have now taken the first two steps in defining a probabilistic model of an experiment with a finite number of outcomes: we have a sample space and a collection

\mathcal{A}

of subsets, which form an algebra and are called events. We now take the next step, to assign each sample point(outcome)

\omega \in \Omega, i = 1, ..., N

, a weight. This is denoted by

p(\omega_i)

and called the ---probability of the outcome---

\omega_i

; we assume that it has the following properties:

$0 \leq p(\omega_i) \leq 1$ (nonnegativity)
$p(\omega_1) + ... + p(\omega_N) = 1$ (normalization)

Starting from the given probabilities

p(\omega_i)

of the outcomes

\omega_i

, we define the probability

P(A)

of any event

A \in \mathcal{A}

by

\begin{equation*}P(A) = \sum_{\{ i: \omega_i \in \mathcal{A} \} } p(\omega_i)\end{equation*}

Finally, we say that a triple (

\Omega, \mathcal{F}, P

), where

\Omega = \{\omega_1, ..., \}

,

\mathcal{A}

is an algebra of subsets of

\Omega

and

P = \{ P(A); A \in \mathcal{A}

defines (or assigns) a ---probabilistic model---, or a --- probability space ---, of experiments with a (finite) space

\Omega

of outcomes and algebra

\mathcal{A}

of events.

The following properties of probability follows:

\begin{equation*}P(\emptyset) = 0; P(\Omega) = 1;\end{equation*}

\begin{equation*}P(A \cup B) = P(A) + P(B) - P(A \cap B)\end{equation*}

General case

The model introduced in the preceding chapter enabled us to give a probabilistic-statistical description of experiments with a finite number of outcomes. \\We now consider the problem of constructing a probabilistic model for the experiment consisting of an infinite number of independent tosses of a coin when at each step the probability of falling head is p. It is natural to take the set of outcomes to be the set

\Omega = \{ \omega: \omega = (a_1, a_2,...), a_i = 0, 1 \},

. What is the cardinality

N(\Omega)

of

\Omega

? It is well known that every number

a \in [0, 1)

has a unique binary expansion (containing an infinite number of zeros). Hence it is clear there is a one-to-one corresponding between the points

\omega \in \Omega

and the points

a

of the set

[0, 1)

, and therefore

\Omega

has the cardinality of the continuum. Consequently, if we wish to construct a probabilistic model to describe experiments like tossing a coin infinitely often, we must consider spaces

\Omega

of a rather complicated nature. \\Since we may take

\Omega

to be the set

[0,1)

, our problem can be considered as the problem of choosing points at random from this set. For reasons of symmetry, it is clear that all outcomes ought to be equiprobable. But the set

[0,1)

is uncountable, and if we suppose that its probability is

1

, then it follows that the probability

p(\omega)

of each outcome certainly must equal zero. However, this assignment of probabilities (

p(\omega)=0,\omega \in [0,1)

) does not lead very far. The fact is that we are ordinarily not interested in the probability of one outcome or another, but in the probability that the result of the experiment is in one or another specified set

A

of outcomes (an event). In elementary probability theory, we use the probabilities

p(\omega)

to find the probability

P(A)

of the event

A : P(A) = \sum_{\omega \in A } p(\omega)

. In the present case, with

p(\omega) = 0

,

\omega \in [0,1)

, we cannot define, for example, the probability that a point chosen at random from

[0,1)

belongs to the set

[0,\frac{1}{2})

. At the same time, it is intuitively clear that this probability should be

\frac{1}{2}

. \\These remarks should suggest that in constructing probabilistic models for uncountable spaces

\Omega

we must assign probabilities, not to individual outcomes but to subsets of

\Omega

. The same reasoning as for the discrete case shows that the collection of sets to which probabilities are assigned must be closed with respect to unions, intersections and complements.Let

\Omega

be a set of points

\omega

. A system

\mathcal{F}

of subsets of

\Omega

is called an

\sigma

-algebra if

$\Omega \in \mathcal{F}$
if $A_n \in \mathcal{F}, n = 1, 2, ...$ , then $\cup A_n \in \mathcal{F}$
if $A \in \mathcal{F} \Rightarrow \hat{A} \in \mathcal{F}$

A measure

\mu

defined on an algebra

\mathcal{A}

of subsets of

\Omega

is countably additive (

\sigma

-additive), or simply a measure, if, for all pairwise disjoint subsets

A_1, A_2, ..

of

\mathcal{A}

with

\sum A_n \in \mathcal{A}

\begin{equation*}\mu(\sum_{n=1}^{\infty} A_n) = \sum_{n=1}^{\infty} \mu (A_n)\end{equation*}

Wiener Space

Brownian motion can be defined in the canonical probability space

(\Omega, \mathcal{F}, P)

known as the Wiener space. More precisely:

$\Omega$ is the space of continuous functions $\omega : \mathbb{R}_{+} \rightarrow \mathbb{R}$ vanishing at the origin.
$\mathcal{F}$ is the Borel $\sigma$ -field $\mathcal{B}(\Omega)$ for the topology corresponding to uniform convergence on compact sets. It can be shown that $\mathcal{F}$ coincides with the $\sigma$ -field generated by the collection of cylinder sets
$P$ is the Wiener measure. That is, $P$ is defined on a cylinder set of the form by

The mapping

P

defined on cylinder sets can be uniquely extended to a probability measure on

\mathcal{F}

. This fact can be proved as a consequence of the existence of Brownian motion on

\mathbb{R}_{+}

. Finally, the canonical stochastic process defined as

B_t (\omega) = \omega (t), \omega \in \Omega, t \geq 0

, is a Brownian motion.

Connection with measure theory

It is clear from the definition that the axiomatic formulation of probability theory is based on set theory and measure theory. Accordingly, it is useful to have a table displaying the ways in which various concepts are interpreted in the two theories.

Notation	Set-theoretic interpretation	Interpretation in probability theory \\
$\omega$	element or point	outcome, sample point, elementary event \\
$\Omega$	set of points	sample space; certain event \\
$\mathcal{F}$	$\sigma$ -algebra of subsets	$\sigma$ -algebra of events \\
$A \in \mathcal{F}$	set of points	event (if $\omega \in A$ , we say that even $A$ occurs) \\
$\overline{A} = \Omega \setminus A$	complement of $A$ , i.e. the set of points $\omega$ that are not in $A$	event that $A$ does not occur \\
$A \cup B$	union of $A$ and $B$ , i.e. the set of points $\omega$ belonging either $A$ or to $B$	event that either $A$ or $B$ occurs \\
$A \cap B$	intersection of $A$ and $B$ , i.e. the set of points $\omega$ belonging to both $A$ and $B$	event that both $A$ and $B$ occurs \\
$\emptyset$	empty set	impossible event \\
$A \cap B = \emptyset$	$A$ and $B$ are disjoint	events $A$ and $B$ are mutually exclusive, i.e. cannot occur simultaneously \\
$A + B$	sum of sets, i.e.union of disjoint sets	event that one of two mutually exclusive events occurs \\
$A \setminus B$	difference of $A$ and $B$ , i.e. the set of points that belong to $A$ but not to $B$	event that $A$ occurs and $B$ does not \\
$A \triangle B$	symmetric difference of sets	event that $A$ or $B$ occurs, but not both \\
$\bigcup^{\infty}_{n=1} A_n$	union of sets $A_1, A_2, ...$	event that at least one $A_1, A_2, ....$ occurs \\
$\bigcap^{\infty}_{n=1} A_n$	intersection of sets $A_1, A_2, ...$	event that all events $A_1, A_2, ....$ occur \\
$\sum^{\infty}_{n=1}$	sum, i.e. union of pairwise disjoint sets $A_1, A_2, ...$	event that one of the mutually exclusive events $A_1, A_2, ....$ occurs \\
$A_n \uparrow A$	the increasing sequence of sets $A_n$ converges to $A$	the increasing sequence of events converges to event $A$ \\
$A_n \downarrow A$	the decreasing sequence of sets $A_n$ converges to $A$	the decreasing sequence of events converges to event $A$ \\
$\overline{\lim} A_n$ or $\lim \sup A_n$ or $\{ A_n \text{i.o.}\}$	the set $\bigcap^{\infty}_{n=1} \bigcup^{\infty}_{k=n} A_k$	event that infinitely many of events $A_1, A_2, ...$ occur \\
$\underline{\lim} A_n$ or $\lim \inf A_n$	the set $\bigcup^{\infty}_{n=1} \bigcap^{\infty}_{k=n} A_k$	event that all the events $A_1, A_2, ...$ occur with the possible exception of a finite number of them\\

References

[MörtersPeres]
Peter Mörters and Yuval Peres. Brownian Motion. Cambridge University Press, 2010.