4 Hitting and Convergence Time, and Flow Rate, Parameters for Reversible Markov Chains (October 11, 1994)

4.5 The flow parameter τc\tau_{c}

In this section it’s convenient to work in continuous time, but the numerical quantities involved here are unchanged by continuization.

4.5.1 Definition and easy inequalities

Define

τc=supAπ(A)π(Ac)Q(A,Ac)\tau_{c}=\sup_{A}\frac{\pi(A)\pi(A^{c})}{Q(A,A^{c})} (4.34)

where

Q(A,Ac)iAjAcπiqijQ(A,A^{c})\equiv\sum_{i\in A}\sum_{j\in A^{c}}\pi_{i}q_{ij}

and where such supssups are always over proper subsets AA of states. This parameter can be calculated exactly in only very special cases, where the following lemma is helpful.

Lemma 4.36

The sup in (4.34) is attained by some split {A,Ac}\{A,A^{c}\} in which both AA and AcA^{c} are connected (as subsets of the graph of permissible transitions).

Proof. Consider a split {A,Ac}\{A,A^{c}\} in which AA is the union of m2m\geq 2 connected components (Bi)(B_{i}). Write γ=miniQ(Bi,Bic)π(Bi)π(Bic)\gamma=\min_{i}\frac{Q(B_{i},B_{i}^{c})}{\pi(B_{i})\pi(B_{i}^{c})}. Then

Q(A,Ac)\displaystyle Q(A,A^{c}) =\displaystyle= iQ(Bi,Bic)\displaystyle\sum_{i}Q(B_{i},B_{i}^{c})
\displaystyle\geq γiπ(Bi)π(Bic)\displaystyle\gamma\sum_{i}\pi(B_{i})\pi(B_{i}^{c})
=\displaystyle= γi(π(Bi)-π2(Bi))\displaystyle\gamma\sum_{i}(\pi(B_{i})-\pi^{2}(B_{i}))
=\displaystyle= γ(π(A)-iπ2(Bi))\displaystyle\gamma\left(\pi(A)-\sum_{i}\pi^{2}(B_{i})\right)

and so

Q(A,Ac)π(A)π(Ac)γπ(A)-iπ2(Bi)π(A)-π2(A).\frac{Q(A,A^{c})}{\pi(A)\pi(A^{c})}\geq\gamma\ \frac{\pi(A)-\sum_{i}\pi^{2}(B_% {i})}{\pi(A)-\pi^{2}(A)}.

But for m2m\geq 2 we have iπ2(Bi)(iπ(Bi))2=π2(A)\sum_{i}\pi^{2}(B_{i})\leq\left(\sum_{i}\pi(B_{i})\right)^{2}=\pi^{2}(A), which implies Q(A,Ac)π(A)π(Ac)>γ\frac{Q(A,A^{c})}{\pi(A)\pi(A^{c})}>\gamma. \Box

To see how τc\tau_{c} arises, note that the extremal characterization of τ2\tau_{2} (4.22) applied to g=1Ag=1_{A} implies

π(A)π(Ac)Q(A,Ac)τ2\frac{\pi(A)\pi(A^{c})}{Q(A,A^{c})}\leq\tau_{2}

for any subset AA. But much more is true: Chapter 3 yyy may be rephrased as follows. For any subset AA,

π(A)π(Ac)Q(A,Ac)π(A)EπTAπ(Ac)π(A)EαATAτ2\frac{\pi(A)\pi(A^{c})}{Q(A,A^{c})}\leq\frac{\pi(A)E_{\pi}T_{A}}{\pi(A^{c})}% \leq\pi(A)E_{\alpha_{A}}T_{A}\leq\tau_{2}

where αA\alpha_{A} is the quasistationary distribution on AcA^{c} defined at Chapter 3 yyy. So taking sups gives

Corollary 4.37
τcsupAπ(A)EπTAπ(Ac)supAπ(A)EαATAτ2.\tau_{c}\leq\sup_{A}\frac{\pi(A)E_{\pi}T_{A}}{\pi(A^{c})}\leq\sup_{A}\pi(A)E_{% \alpha_{A}}T_{A}\leq\tau_{2}.

In a two-state chain these inequalities all become equalities. This seems a good justification for our choice of definition of τc\tau_{c}, instead of the alternative

supA:π(A)1/2π(A)Q(A,Ac)\sup_{A:\pi(A)\leq 1/2}\frac{\pi(A)}{Q(A,A^{c})}

which has been used in the literature but which would introduce a spurious factor of 22 into the inequality τcτ2\tau_{c}\leq\tau_{2}.

Lemma 4.39 below shows that the final inequality of Corollary 4.37 can be reversed. In contrast, on the nn-cycle τc=Θ(n)\tau_{c}=\Theta(n) whereas the other quantities in Corollary 4.37 are Θ(n2)\Theta(n^{2}). This shows that the “square” in Theorem 4.40 below cannot be omitted in general. It also suggests the following question (cf. τ1\tau_{1} and τ1(5)\tau^{(5)}_{1})

Open Problem 4.38

Does there exist a constant KK such that

τ2KsupAπ(A)EπTAπ(Ac)\tau_{2}\leq K\sup_{A}\frac{\pi(A)E_{\pi}T_{A}}{\pi(A^{c})}

for every reversible chain?

A positive answer would provide, via Chapter 3 yyy, a correct order-of-magnitude extremal characterization of τ2\tau_{2} in terms of flows.

Lemma 4.39
τ2supA:π(A)1/2EαATA\tau_{2}\leq\sup_{A:\pi(A)\geq 1/2}E_{\alpha_{A}}T_{A}

and so in particular

τ22supAπ(A)EαATA.\tau_{2}\leq 2\sup_{A}\pi(A)E_{\alpha_{A}}T_{A}.

Proof. τ2=||h||22/(h,h)\tau_{2}=||h||_{2}^{2}/\mbox{${\cal E}$}(h,h) for the eigenvector hh associated with λ2\lambda_{2}. Put

A={x:h(x)0}A=\{x:h(x)\leq 0\}

and assume π(A)1/2\pi(A)\geq 1/2, by replacing hh by -h-h if necessary. Write h+=max(h,0)h_{+}=\max(h,0). We shall show

τ2||h+||22/(h+,h+)\tau_{2}\leq||h_{+}||_{2}^{2}/\mbox{${\cal E}$}(h_{+},h_{+}) (4.35)

and then the extremal characterization Chapter 3 yyy

EαATA=sup{||g||22/(g,g):g0,g=0 on A}E_{\alpha_{A}}T_{A}=\sup\{||g||_{2}^{2}/\mbox{${\cal E}$}(g,g):g\geq 0,g=0% \mbox{ on }A\} (4.36)

implies τ2EαATA\tau_{2}\leq E_{\alpha_{A}}T_{A} for this specific AA.

The proof of (4.35) requires us to delve slightly further into the calculus of Dirichlet forms. Write 𝐏tf{\bf P}_{t}f for the function (𝐏tf)(i)=Eif(Xt)({\bf P}_{t}f)(i)=E_{i}f(X_{t}) and write f,g\langle f,g\rangle for the inner product iπif(i)g(i)\sum_{i}\pi_{i}f(i)g(i). Write ()\partial(\cdot) for ddt()t=0\frac{d}{dt}(\cdot)_{t=0}. Then

f,𝐏tg=-(f,g)\partial\langle f,{\bf P}_{t}g\rangle=-\mbox{${\cal E}$}(f,g)

where

(f,g)=12ij(f(j)-f(i))(g(j)-g(i))qij.\mbox{${\cal E}$}(f,g)=\frac{1}{2}\ \sum_{i}\sum_{j}(f(j)-f(i))(g(j)-g(i))q_{% ij}.

Now consider h+,𝐏th\partial\langle h_{+},{\bf P}_{t}h\rangle. On the one hand

h+,𝐏th=-(h+,h)-(h+,h+)\partial\langle h_{+},{\bf P}_{t}h\rangle=-\mbox{${\cal E}$}(h_{+},h)\leq-% \mbox{${\cal E}$}(h_{+},h_{+})

where the inequality follows from the inequality (a+-b+)2(a+-b+)(a-b)(a^{+}-b^{+})^{2}\leq(a^{+}-b^{+})(a-b) for real a,ba,b. On the other hand, h+,hh+,h+=||h+||22\langle h_{+},h\rangle\leq\langle h_{+},h_{+}\rangle=||h_{+}||_{2}^{2}, and the eigenvector hh satisfies (𝐏th)=-λ2h\partial({\bf P}_{t}h)=-\lambda_{2}h, so

h+,𝐏th-λ2||h+||22.\partial\langle h_{+},{\bf P}_{t}h\rangle\geq-\lambda_{2}||h_{+}||_{2}^{2}.

Combining these inequalities leads to (4.35).

4.5.2 Cheeger-type inequalities

A lot of attention has been paid to reverse inequalities which upper bound τ2\tau_{2} in terms of τc\tau_{c} or related “flow rate” parameters. Motivation for such results will be touched upon in Chapter yyy. The prototype for such results is

Theorem 4.40 (Cheeger’s inequality)

τ28q*τc2\tau_{2}\leq 8q^{*}\tau^{2}_{c}, where q*maxiqiq^{*}\equiv\max_{i}q_{i}.

This result follows by combining Lemma 4.39 above with Lemma 4.41 below. In discrete time these inequalities hold with q*q^{*} deleted (i.e. replaced by 11), by continuization. Our treatment of Cheeger’s inequality closely follows Diaconis and Stroock [122] – see Notes for more history.

Lemma 4.41

For any subset AA,

EαATA2q*τc2π2(A).E_{\alpha_{A}}T_{A}\leq\frac{2q^{*}\tau^{2}_{c}}{\pi^{2}(A)}.

Proof. Fix AA and gg with g0g\geq 0 and g=0g=0 on AA.

(xy|g2(x)-g2(y)|πxqxy)2\displaystyle\left(\sum\sum_{x\neq y}|g^{2}(x)-g^{2}(y)|\pi_{x}q_{xy}\right)^{2}
\displaystyle\leq xy(g(x)-g(y))2πxqxy×xy(g(x)+g(y))2πxqxy\displaystyle\sum\sum_{x\neq y}(g(x)-g(y))^{2}\pi_{x}q_{xy}\times\sum\sum_{x% \neq y}(g(x)+g(y))^{2}\pi_{x}q_{xy}
by the Cauchy-Schwarz inequality
=\displaystyle= 2(g,g)xy(g(x)+g(y))2πxqxy\displaystyle 2\mbox{${\cal E}$}(g,g)\sum\sum_{x\neq y}(g(x)+g(y))^{2}\pi_{x}q% _{xy}
\displaystyle\leq 4(g,g)xy(g2(x)+g2(y))πxqxy\displaystyle 4\mbox{${\cal E}$}(g,g)\sum\sum_{x\neq y}(g^{2}(x)+g^{2}(y))\pi_% {x}q_{xy}
=\displaystyle= 8(g,g)xπxqxg2(x)\displaystyle 8\mbox{${\cal E}$}(g,g)\sum_{x}\pi_{x}q_{x}g^{2}(x)
\displaystyle\leq 8q*(g,g)||g||22.\displaystyle 8q^{*}\mbox{${\cal E}$}(g,g)\ ||g||_{2}^{2}.

On the other hand

xy|g2(x)-g2(y)|πxqxy\displaystyle\sum\sum_{x\neq y}|g^{2}(x)-g^{2}(y)|\pi_{x}q_{xy}
=\displaystyle= 2g(x)>g(y)(g2(x)-g2(y))πxqxy\displaystyle 2\sum\sum_{g(x)>g(y)}(g^{2}(x)-g^{2}(y))\pi_{x}q_{xy}
=\displaystyle= 4g(x)>g(y)(g(y)g(x)tdt)πxqxy\displaystyle 4\sum\sum_{g(x)>g(y)}\left(\int_{g(y)}^{g(x)}tdt\right)\pi_{x}q_% {xy}
=\displaystyle= 40t(g(y)t<g(x)πxqxy)dt\displaystyle 4\int_{0}^{\infty}t\ \left(\sum\sum_{g(y)\leq t<g(x)}\pi_{x}q_{% xy}\right)\ dt
=\displaystyle= 40tQ(Bt,Btc)dt where Bt{x:g(x)>t}\displaystyle 4\int_{0}^{\infty}t\ Q(B_{t},B^{c}_{t})\ dt\mbox{ where }B_{t}% \equiv\{x:g(x)>t\}
\displaystyle\geq 40tπ(Bt)π(Btc)τcdt by definition of τc\displaystyle 4\int_{0}^{\infty}t\ \frac{\pi(B_{t})\ \pi(B^{c}_{t})}{\tau_{c}}% \ dt\mbox{ by definition of }\tau_{c}
\displaystyle\geq 40tπ(Bt)π(A)τcdt because g=0 on A\displaystyle 4\int_{0}^{\infty}t\ \frac{\pi(B_{t})\ \pi(A)}{\tau_{c}}\ dt% \mbox{ because }g=0\mbox{ on }A
=\displaystyle= 2π(A)||g||22τc.\displaystyle\frac{2\pi(A)||g||_{2}^{2}}{\tau_{c}}.

Rearranging,

||g||22(g,g)2q*τc2π2(A)\frac{||g||_{2}^{2}}{\mbox{${\cal E}$}(g,g)}\leq\frac{2q^{*}\tau^{2}_{c}}{\pi^% {2}(A)}

and the first assertion of the Theorem follows from the extremal characterization (4.36) of EαATAE_{\alpha_{A}}T_{A}.

4.5.3 τc\tau_{c} and hitting times

Lemma 4.25 and Theorem 4.40 imply a bound on τ*\tau^{*} in terms of τc\tau_{c}. But a direct argument, using ideas similar to those in the proof of Lemma 4.41, does better.

Proposition 4.42
τ*4(1+logn)minjπjτc.\tau^{*}\leq\frac{4(1+\log n)}{\min_{j}\pi_{j}}\ \tau_{c}.

Example 4.43 below will show that the log term cannot be omitted. Compare with graph-theoretic bounds in Chapter 6 section yyy.

Proof. Fix states a,ba,b. We want to use the extremal characterization (Chapter 3 yyy). So fix a function 0g10\leq g\leq 1 with g(a)=0,g(b)=1g(a)=0,g(b)=1. Order the states as a=1,2,3,,n=ba=1,2,3,\ldots,n=b so that g()g(\cdot) is increasing.

(g,g)\displaystyle\mbox{${\cal E}$}(g,g) =\displaystyle= i<kπiqik(g(k)-g(i))2\displaystyle\sum\sum_{i<k}\pi_{i}q_{ik}(g(k)-g(i))^{2} (4.37)
\displaystyle\geq ij<kπiqik(g(j+1)-g(j))2\displaystyle\sum\sum\sum_{i\leq j<k}\pi_{i}q_{ik}(g(j+1)-g(j))^{2}
=\displaystyle= j(g(j+1)-g(j))2Q(Aj,Ajc), where Aj[1,j]\displaystyle\sum_{j}(g(j+1)-g(j))^{2}Q(A_{j},A^{c}_{j}),\mbox{ where }A_{j}% \equiv[1,j]
\displaystyle\geq j(g(j+1)-g(j))2π(Aj)π(Ajc)τc\displaystyle\sum_{j}(g(j+1)-g(j))^{2}\frac{\pi(A_{j})\pi(A^{c}_{j})}{\tau_{c}}

But

1=j(g(j+1)-g(j))=j(g(j+1)-g(j))π1/2(Aj)π1/2(Ajc)τc1/2τc1/2π1/2(Aj)π1/2(Ajc).1=\sum_{j}(g(j+1)-g(j))=\sum_{j}(g(j+1)-g(j))\ \frac{\pi^{1/2}(A_{j})\pi^{1/2}% (A^{c}_{j})}{\tau_{c}^{1/2}}\ \frac{\tau_{c}^{1/2}}{\pi^{1/2}(A_{j})\pi^{1/2}(% A^{c}_{j})}.

So by Cauchy-Schwarz and (4.37)

1τc(g,g)j1π(Aj)π(Ajc).1\leq\tau_{c}\mbox{${\cal E}$}(g,g)\sum_{j}\frac{1}{\pi(A_{j})\pi(A^{c}_{j})}. (4.38)

But π(Aj)jπ*\pi(A_{j})\geq j\pi_{*}, where π*miniπi\pi_{*}\equiv\min_{i}\pi_{i}, so

j:π(Aj)1/21π(Aj)π(Ajc)j2jπ*2π*(1+logn).\sum_{j:\pi(A_{j})\leq 1/2}\frac{1}{\pi(A_{j})\pi(A^{c}_{j})}\leq\sum_{j}\frac% {2}{j\pi_{*}}\leq\frac{2}{\pi_{*}}(1+\log n).

The same bound holds for the sum over {j:π(Aj)1/2}\{j:\pi(A_{j})\geq 1/2\}, so applying (4.38) we get

1(g,g)τc4π*(1+logn)\frac{1}{\mbox{${\cal E}$}(g,g)}\leq\tau_{c}\ \frac{4}{\pi_{*}}(1+\log n)

and the Proposition follows from the extremal characterization.

Example 4.43

Consider the weighted linear graph with loops on vertices {0,1,2,,n-1}\{0,1,2,\ldots,n-1\}, with edge-weights

wi-1,i=i, 1in-1;  wii=2n-i1(i0)-(i+1)1(in-1).w_{i-1,i}=i,\ 1\leq i\leq n-1;\ \ \ \ w_{ii}=2n-i1_{(i\neq 0)}-(i+1)1_{(i\neq n% -1)}.

This gives vertex-weights wi=2nw_{i}=2n, and so the stationary distribution is uniform. By the commute interpretation of resistance,

τ*=E0Tn-1+En-1T0=wr0,n-1=2n2i=1n-11/i2n2logn.\tau^{*}=E_{0}T_{n-1}+E_{n-1}T_{0}=wr_{0,n-1}=2n^{2}\sum_{i=1}^{n-1}1/i\sim 2n% ^{2}\log n.

Using Lemma 4.36, the value of τc\tau_{c} is attained by a split of the form {[0,j],[j+1,n-1]}\{[0,j],[j+1,n-1]\}, and a brief calculation shows that the maximizing value is j=0j=0 and gives

τc=2(n-1).\tau_{c}=2(n-1).

So in this example, the bound in Proposition 4.42 is sharp up to the numerical constant.