16 1. Preparatory material

(iv) X has sub-exponential tail if there exist C, c, a 0 such that

P(|X| ≥ λ) ≤ C

exp(−cλa)

for all λ 0.

(v) X has finite

kth

moment for some k ≥ 1 if there exists C such that

E|X|k

≤ C.

(vi) X is absolutely integrable if E|X| ∞.

(vii) X is almost surely finite if |X| ∞ almost surely.

Exercise 1.1.2. Show that these properties genuinely are in decreasing

order of strength, i.e., that each property on the list implies the next.

Exercise 1.1.3. Show that each of these properties are closed under vector

space operations, thus, for instance, if X, Y have sub-exponential tail, show

that X + Y and cX also have sub-exponential tail for any scalar c.

Examples 1.1.9. The various species of Bernoulli random variable are

surely bounded, and any random variable which is uniformly distributed

in a bounded set is almost surely bounded. Gaussians and Poisson dis-

tributions are sub-Gaussian, while the geometric distribution merely has

sub-exponential tail. Cauchy distributions (which have density functions of

the form f(x) =

1

π

γ

(x−x0)2+γ2

) are typical examples of heavy-tailed distri-

butions which are almost surely finite, but do not have all moments finite

(indeed, the Cauchy distribution does not even have finite first moment).

If we have a family of scalar random variables Xα depending on a pa-

rameter α, we say that the Xα are uniformly surely bounded (resp. uni-

formly almost surely bounded, uniformly sub-Gaussian, have uniform sub-

exponential tails, or uniformly bounded

kth

moment) if the relevant param-

eters M, C, c, a in the above definitions can be chosen to be independent of

α.

Fix k ≥ 1. If X has finite

kth

moment, say

E|X|k

≤ C, then from

Markov’s inequality (1.14) one has

(1.23) P(|X| ≥ λ) ≤

Cλ−k,

thus we see that the higher the moments that we control, the faster the tail

decay is. From the dominated convergence theorem we also have the variant

(1.24) lim

λ→∞

λkP(|X|

≥ λ) = 0.

However, this result is qualitative or ineffective rather than quantitative

because it provides no rate of convergence of

λkP(|X|

≥ λ) to zero. Indeed,

it is easy to construct a family Xα of random variables of uniformly bounded

kth

moment, but for which the quantities

λkP(|Xα|

≥ λ) do not converge

uniformly to zero (e.g., take Xm to be m times the indicator of an event of

probability

m−k

for m = 1, 2,...). Because of this issue, we will often have