1.1. A review of probability theory 19

However, once one studies families (Xα)α∈A of random variables Xα

taking values in measurable spaces Rα (on a single sample space Ω), the

distribution of the individual variables Xα are no longer suﬃcient to de-

scribe all the probabilistic statistics of interest; the joint distribution of the

variables (i.e., the distribution of the tuple (Xα)α∈A, which can be viewed

as a single random variable taking values in the product measurable space

α∈A

Rα) also becomes relevant.

Example 1.1.11. Let (X1,X2) be drawn uniformly at random from the

set {(−1, −1), (−1, +1), (+1, −1), (+1, +1)}. Then the random variables

X1, X2, and −X1 all individually have the same distribution, namely the

signed Bernoulli distribution. However, the pairs (X1,X2), (X1,X1), and

(X1, −X1) all have different joint distributions: the first pair, by definition,

is uniformly distributed in the set

{(−1, −1), (−1, +1), (+1, −1), (+1, +1)},

while the second pair is uniformly distributed in {(−1, −1), (+1, +1)}, and

the third pair is uniformly distributed in {(−1, +1), (+1, −1)}. Thus, for

instance, if one is told that X, Y are two random variables with the Bernoulli

distribution, and asked to compute the probability that X = Y , there is

insuﬃcient information to solve the problem; if (X, Y ) were distributed as

(X1,X2), then the probability would be 1/2, while if (X, Y ) were distributed

as (X1,X1), the probability would be 1, and if (X, Y ) were distributed as

(X1, −X1), the probability would be 0. Thus one sees that one needs the

joint distribution, and not just the individual distributions, to obtain a

unique answer to the question.

There is, however, an important special class of families of random vari-

ables in which the joint distribution is determined by the individual distri-

butions.

Definition 1.1.12 (Joint independence). A family (Xα)α∈A of random vari-

ables (which may be finite, countably infinite, or uncountably infinite) is

said to be jointly independent if the distribution of (Xα)α∈A is the product

measure of the distribution of the individual Xα.

A family (Xα)α∈A is said to be pairwise independent if the pairs (Xα,Xβ)

are jointly independent for all distinct α, β ∈ A. More generally, (Xα)α∈A

is said to be k-wise independent if (Xα1 , . . . , Xαk ) are jointly independent

for all 1 ≤ k ≤ k and all distinct α1,...,αk ∈ A.

We also say that X is independent of Y if (X, Y ) are jointly independent.

A family of events (Eα)α∈A is said to be jointly independent if their

indicators (I(Eα))α∈A are jointly independent. Similarly for pairwise inde-

pendence and k-wise independence.