1.1. A review of probability theory 25

refer to this conditioned random variable as (X|E), and thus define con-

ditional distribution μ(X|E) and conditional expectation E(X|E) (if X is

scalar) accordingly.

One can also condition on the complementary event E, provided that

this event holds with positive probility also.

By undoing this conditioning, we revert the underlying sample space

and measure back to their original (or unconditional) values. Note that any

random variable which has been defined both after conditioning on E, and

conditioning on E, can still be viewed as a combined random variable after

undoing the conditioning.

Conditioning affects the underlying probability space in a manner which

is different from extension, and so the act of conditioning is not guaranteed

to preserve probabilistic concepts such as distribution, probability, or expec-

tation. Nevertheless, the conditioned version of these concepts are closely

related to their unconditional counterparts:

Exercise 1.1.21. If E and E both occur with positive probability, establish

the identities

(1.32) P(F ) = P(F |E)P(E) + P(F |E)P(E)

for any (unconditional) event F and

(1.33) μX = μ(X|E)P(E) + μ(X|E)P(E)

for any (unconditional) random variable X (in the original sample space).

In a similar spirit, if X is a non-negative or absolutely integrable scalar

(unconditional) random variable, show that (X|E), (X|E) are also non-

negative and absolutely integrable on their respective conditioned spaces,

and that

(1.34) EX = E(X|E)P(E) + E(X|E)P(E).

In the degenerate case when E occurs with full probability, conditioning to

the complementary event E is not well defined, but show that in those cases

we can still obtain the above formulae if we adopt the convention that any

term involving the vanishing factor P(E) should be omitted. Similarly if E

occurs with zero probability.

The above identities allow one to study probabilities, distributions, and

expectations on the original sample space by conditioning to the two condi-

tioned spaces.

From (1.32) we obtain the inequality

(1.35) P(F |E) ≤ P(F )/P(E),