1.5. STATIONARY DISTRIBUTIONS 13 The expression in (1.22) is very similar to (1.19), so we are almost done. In fact, t=1 Pz{Xt = y, τz + t} = ˜(y) π Pz{X0 = y, τz + 0} + t=1 Pz{Xt = y, τz + = t} = ˜(y) π Pz{X0 = y} + Pz{Xτz + = y}. (1.23) = ˜(y). π (1.24) The equality (1.24) follows by considering two cases: y = z: Since X0 = z and Xτz + = z, the last two terms of (1.23) are both 1, and they cancel each other out. y = z: Here both terms of (1.23) are 0. Therefore, combining (1.22) with (1.24) shows that ˜ π = ˜ πP . Finally, to get a probability measure, we normalize by x ˜(x) π = Ez(τz +): π(x) = ˜(x) π Ez(τz +) satisfies π = πP. (1.25) In particular, for any x Ω, π(x) = 1 Ex(τx + ) . (1.26) The computation at the heart of the proof of Proposition 1.14 can be general- ized. A stopping time τ for (Xt) is a {0, 1, . . . , } {∞}-valued random variable such that, for each t, the event = t} is determined by X0, . . . , Xt. (Stopping times are discussed in detail in Section 6.2.1.) If a stopping time τ replaces τz + in the definition (1.19) of ˜, π then the proof that ˜ π satisfies ˜ π = ˜ πP works, provided that τ satisfies both Pz{τ ∞} = 1 and Pz{Xτ = z} = 1. If τ is a stopping time, then an immediate consequence of the definition and the Markov property is Px0 {(Xτ+1, Xτ+2, . . . , X ) A | τ = k and (X1, . . . , Xk) = (x1, . . . , xk)} = Pxk {(X1, . . . , X ) A}, (1.27) for any A . This is referred to as the strong Markov property. Informally, we say that the chain “starts afresh” at a stopping time. While this is an easy fact for countable state space, discrete-time Markov chains, establishing it for processes in the continuum is more subtle. 1.5.4. Uniqueness of the stationary distribution. Earlier this chapter we pointed out the difference between multiplying a row vector by P on the right and a column vector by P on the left: the former advances a distribution by one step of the chain, while the latter gives the expectation of a function on states, one step of the chain later. We call distributions invariant under right multiplication by P stationary. What about functions that are invariant under left multiplication? Call a function h : R harmonic at x if h(x) = y∈Ω P (x, y)h(y). (1.28)
Previous Page Next Page