14 1. Preparatory material
By the Fubini-Tonelli theorem, the same result also applies to infinite sums
ciXi provided that
|ci|E|Xi| is finite.
We will use linearity of expectation so frequently in the sequel that we
will often omit an explicit reference to it when it is being used. It is im-
portant to note that linearity of expectation requires no assumptions of
amongst the individual random variables Xi;
this is what makes this property of expectation so powerful.
In the unsigned (or real absolutely integrable) case, expectation is also
monotone: if X ≤ Y is true for some unsigned or real absolutely integrable
X, Y , then EX ≤ EY . Again, we will usually use this basic property without
explicit mentioning it in the sequel.
For an unsigned random variable, we have the obvious but very useful
(1.13) P(X ≥ λ) ≤
for any λ 0, as can be seen by taking expectations of the inequality
λI(X ≥ λ) ≤ X. For signed random variables, Markov’s inequality becomes
(1.14) P(|X| ≥ λ) ≤
Another fact related to Markov’s inequality is that if X is an unsigned
or real absolutely integrable random variable, then X ≥ EX must hold
with positive probability, and also X ≤ EX must also hold with positive
probability. Use of these facts or (1.13), (1.14), combined with monotonicity
and linearity of expectation, is collectively referred to as the first moment
method. This method tends to be particularly easy to use (as one does not
need to understand dependence or independence), but by the same token
often gives sub-optimal results (as one is not exploiting any independence
in the system).
Exercise 1.1.1 (Borel-Cantelli lemma). Let E1,E2,... be a sequence of
events such that
P(Ei) ∞. Show that almost surely, at most finitely
many of the events Ei occur at once. State and prove a result to the effect
that the condition
P(Ei) ∞ cannot be weakened.
If X is an absolutely integrable or unsigned scalar random variable, and
F is a measurable function from the scalars to the unsigned extended reals
[0, +∞], then one has the change of variables formula
(1.15) EF (X) =
F (x) dμX(x)
will define these terms in Section 1.1.3.