1.1. A review of probability theory 19
However, once one studies families (Xα)α∈A of random variables
taking values in measurable spaces (on a single sample space Ω), the
distribution of the individual variables are no longer sufficient to de-
scribe all the probabilistic statistics of interest; the joint distribution of the
variables (i.e., the distribution of the tuple (Xα)α∈A, which can be viewed
as a single random variable taking values in the product measurable space
α∈A
Rα) also becomes relevant.
Example 1.1.11. Let (X1,X2) be drawn uniformly at random from the
set {(−1, −1), (−1, +1), (+1, −1), (+1, +1)}. Then the random variables
X1, X2, and −X1 all individually have the same distribution, namely the
signed Bernoulli distribution. However, the pairs (X1,X2), (X1,X1), and
(X1, −X1) all have different joint distributions: the first pair, by definition,
is uniformly distributed in the set
{(−1, −1), (−1, +1), (+1, −1), (+1, +1)},
while the second pair is uniformly distributed in {(−1, −1), (+1, +1)}, and
the third pair is uniformly distributed in {(−1, +1), (+1, −1)}. Thus, for
instance, if one is told that X, Y are two random variables with the Bernoulli
distribution, and asked to compute the probability that X = Y , there is
insufficient information to solve the problem; if (X, Y ) were distributed as
(X1,X2), then the probability would be 1/2, while if (X, Y ) were distributed
as (X1,X1), the probability would be 1, and if (X, Y ) were distributed as
(X1, −X1), the probability would be 0. Thus one sees that one needs the
joint distribution, and not just the individual distributions, to obtain a
unique answer to the question.
There is, however, an important special class of families of random vari-
ables in which the joint distribution is determined by the individual distri-
butions.
Definition 1.1.12 (Joint independence). A family (Xα)α∈A of random vari-
ables (which may be finite, countably infinite, or uncountably infinite) is
said to be jointly independent if the distribution of (Xα)α∈A is the product
measure of the distribution of the individual Xα.
A family (Xα)α∈A is said to be pairwise independent if the pairs (Xα,Xβ)
are jointly independent for all distinct α, β A. More generally, (Xα)α∈A
is said to be k-wise independent if (Xα1 , . . . , Xαk ) are jointly independent
for all 1 k k and all distinct α1,...,αk A.
We also say that X is independent of Y if (X, Y ) are jointly independent.
A family of events (Eα)α∈A is said to be jointly independent if their
indicators (I(Eα))α∈A are jointly independent. Similarly for pairwise inde-
pendence and k-wise independence.
Previous Page Next Page