16 1. Preparatory material
(iv) X has sub-exponential tail if there exist C, c, a 0 such that
P(|X| ≥ λ) ≤ C
for all λ 0.
(v) X has finite
moment for some k ≥ 1 if there exists C such that
(vi) X is absolutely integrable if E|X| ∞.
(vii) X is almost surely finite if |X| ∞ almost surely.
Exercise 1.1.2. Show that these properties genuinely are in decreasing
order of strength, i.e., that each property on the list implies the next.
Exercise 1.1.3. Show that each of these properties are closed under vector
space operations, thus, for instance, if X, Y have sub-exponential tail, show
that X + Y and cX also have sub-exponential tail for any scalar c.
Examples 1.1.9. The various species of Bernoulli random variable are
surely bounded, and any random variable which is uniformly distributed
in a bounded set is almost surely bounded. Gaussians and Poisson dis-
tributions are sub-Gaussian, while the geometric distribution merely has
sub-exponential tail. Cauchy distributions (which have density functions of
the form f(x) =
) are typical examples of heavy-tailed distri-
butions which are almost surely finite, but do not have all moments finite
(indeed, the Cauchy distribution does not even have finite first moment).
If we have a family of scalar random variables Xα depending on a pa-
rameter α, we say that the Xα are uniformly surely bounded (resp. uni-
formly almost surely bounded, uniformly sub-Gaussian, have uniform sub-
exponential tails, or uniformly bounded
moment) if the relevant param-
eters M, C, c, a in the above definitions can be chosen to be independent of
Fix k ≥ 1. If X has finite
≤ C, then from
Markov’s inequality (1.14) one has
(1.23) P(|X| ≥ λ) ≤
thus we see that the higher the moments that we control, the faster the tail
decay is. From the dominated convergence theorem we also have the variant
≥ λ) = 0.
However, this result is qualitative or ineffective rather than quantitative
because it provides no rate of convergence of
≥ λ) to zero. Indeed,
it is easy to construct a family Xα of random variables of uniformly bounded
moment, but for which the quantities
≥ λ) do not converge
uniformly to zero (e.g., take Xm to be m times the indicator of an event of
for m = 1, 2,...). Because of this issue, we will often have