# Specific question to a Markov chain proof in Durrett

I apologize if this is to specific but i've already talked to two of my professors without much success and I really need to understand this subject. The following theorem is stated in Durrett page 254

Let x be a recurrent state, and let $T=\inf \{n\geq 1: X_n =x\}$ then $$\mu _x (y) = \sum_{n=0} ^\infty P_x (X_n =y, T > n)$$ defines a stationary measure.

He proves it and then writes a "technical note" saying that "To show we are not cheating, we should prove that $\mu _x (y)<\infty$". Why do we need that? One of my professors (who is usually right about things) talked about it being impossible for a stationary measure to give infinite mass to a point (I'm not quite sure what mass means in that context), but I really can't find any reason (in the book anyway) for that to be true.

I could really need to ask a few short questions to a person familiar with Durrett and his chapter about Markov chains.

p.s. I have the relevant pages as a pdf which I can supply if anyone would like.

Henrik

-
Can you provide the definition of stationnary measure ? Light could come from it. ;) –  Ahriman Sep 6 '12 at 16:21
@Ahriman: $\nu\cdot p=\nu$. –  Did Sep 6 '12 at 16:37
@did That was a pedagogical question, I know the answer ... Anyway, I was thinking that the fact a stationnary measure doesn't give infinite mass to a singleton was part of the definition, but apparently this is not explicitely said in Durrett's book. –  Ahriman Sep 6 '12 at 17:10

Let $\nu$ denote a stationary measure. If $\nu(y)$ is infinite for some state $y$, then, for every state $z$ such that $p(y,z)\ne0$, the inequality $\nu(z)\geqslant\nu(y) p(y,z)$ shows that $\nu(z)$ is infinite as well. If the Markov chain is irreducible, this proves that $\nu(z)$ is infinite for every state $z$. This measure $\nu$ is stationary but not very interesting. What Durrett says is that, to prove that the fancy formula he proposes for $\mu_x$ does not lead to an empty statement, one should check that $\mu_x(y)$ is finite for some state $y$ (or, equivalently when the chain is irreducible, for every state $y$).

(The mass of a measure $\nu$ at a point $y$ is $\nu(\{y\})$, often denoted by $\nu(y)$ when $\nu$ is discrete, by an abuse of notation.)

-
That is perfect! Thank you very much. I noticed you replied arihman that being stationary means ν⋅p=ν, and I have been puzzled with that notation too. I guess it has to be $sum_x \mu(x) p(x,y) = \mu(y)$ - since that is how he introduces stationary, but he later (282, general state space harris chains) wants to show vp=p where v and p are transition probabilities and even says that it means to show that $\mu v p = \mu p$. So to sum up: what does it mean to multiply measure and transition function or two transition functions? –  Henrik Sep 6 '12 at 17:13
It all depends on the size of the objects: vp=v makes sense because, as a measure on a discrete space of size n, v has size 1xn and, as a transition matrix on the same space, p has size nxn, hence vp has size 1xn, that is, vp is a measure just like v. On the other hand, if p and q are nxn transition matrices, so is qp hence the relation qp=p makes sense. Finally, qp=p if and only if vqp=vp for every measure (1xn vector) v. –  Did Sep 6 '12 at 18:19
Ahh of course! Studying Durrett for so long made me totally forget the good old representation by matrices. I still wonder how this works in the general state space though - just integrals? You have no idea how much you've helped me, I've really been wondering about this. –  Henrik Sep 6 '12 at 18:42
Pretty much: measures instead of vectors, kernels instead of matrices, integration instead of matrix-multiplication, et voilà ! –  Did Sep 6 '12 at 19:19

Since x is a recurrent state it occurs infinitely often. Does Durrett say whether or not x is positive recurrent? If it is positive recurrent the average return time is finite. If it is null recurrent the average return time is infinite even though the state is recurrent. In the positive recurrent case, T is guaranteed to be finite. So there will only be some finite number of values for n that will be less than T. For each nn) is the probability of transitioning from y to x in K$_n$ steps. This is a finite probability.

for each n P$_x$(X$_n$=y, for T>n)=∑P$_x$(X$_n$=y, k>n|T=k) P(T=k) where the sum is taken over k>=n+1. At this point I think positive recurrence is necessary and should imply that P(T=k) goes to 0 quickly enough as k approaches infinity so that the sum on the right hand side of the equation is finite and approaches 0 as n approaches infinity.

You need P$_x$(X$_n$=y, for T>n) to go to 0 sufficiently fast so that the sum that produces μ$_x$(y) is finite.

This is sketchy I know but I hope that someone else can pick it up and fill in the details for you. I think that presenting this at least gives you an idea as to why the measure is finite and also that if certain quantities do not go to 0 sufficiently fast (as I think would happen in the null recurrent case) the measure could be infinite.

-
Your answer did help, thank you. Your reply actually looks a lot like how he proves it is finite - what I was wondering though wasn't as much how to prove as to how though (I know the two are interconnected), –  Henrik Sep 6 '12 at 17:16
@Henrik Did Rick state that the state was positive recurrent in the statement of the theorem? –  Michael Chernick Sep 6 '12 at 17:28
Nope. He actually haven't even defined positive recurrence when stating the theorem. –  Henrik Sep 6 '12 at 17:31
@Henrik So then does the proof go through even for null recurrence? –  Michael Chernick Sep 6 '12 at 17:34
A quick sketch: First observe that $\mu _x p= \mu _x$ implies $\mu _x p^n= \mu _x$ and $\mu _x (x)=1$ per definition. But that means that if $p^n(y,x)>0$ then $\mu_x(y)<\infty$. But since the above is true for all n it is true whenever the chain has positive probability of transition in finite time. Now the dichotomy of recurrent states makes sure it is true for all states which positive probability of being reached in finite time. –  Henrik Sep 6 '12 at 17:43