We quote a result, Theorem 2.36, which may look superficially like the identities in section 2.2.1 but which in fact is deeper, in that it cannot be proved by mere matrix manipulations or by Proposition 2.3. The result goes back to Baxter and Chacon [44] (and is implicit in Rost [301]) in the more general continuous-space setting: a proof tailored to the finite state space case has recently been given by Lovász and Winkler [240].
Given distributions , consider a stopping time such that
(2.29) |
Clearly, for any state we have , which rearranges to . So if we define
then we have shown that . Surprisingly, this inequality turns out to be an equality.
.
From the definition (2.6) of the fundamental matrix we can write, in matrix notation,
(2.30) |
where is the matrix with -entry . The matrix is not invertible but (2.30) expresses as a “generalized inverse” of , and one can use matrix methods to verify general identities in the spirit of section 2.2.1. See e.g. [186, 215]. Here is a setting where such matrix methods work well.
Suppose (and hence and ) depend on a real parameter , and suppose exists. Then, at such that is irreducible,
Proof. Write . Differentiating the balance equations gives , in other words . Right-multiply by to get
But because .