# A relation between first passage time and occupation time

Let's think about a discrete time Markov chain $X_t$ with only one recurrent state. Let $T$ be the random variable that is the number of steps taken from a given state $i$ to the recurrent state (ie. the first passage time to the recurrent state) and $\Phi_k(n) = \sum_{t=0}^n I(X_t = k | X_0 = i)$ be the random variable of occupation time of state $k$ during $n$ steps, where $I$ is the indicator function. Today, in a class on stochastic process, I learned a relation between them : $T=\sum_k \Phi_k(T)$. I'm very confused with this. Is it really legal? How the same random variable $T$ can be appeared in two places in an equation? I have no idea how to interpret the relation. Could someone explain it in plain English?

Yes, this is legal. The right-hand side is a function of a random variable, which again yields a random variable, and the equation states that the random variables on its two sides are equal, which is also a meaningful statement.

The statement isn't quite correct, however; the left-hand side should be $T+1$, or the summation on the right-hand side should be restricted to the non-recurrent states. The process was in exactly one state at any given time, so the sum on the right-hand side contains exactly $T+1$ ones.

• You are right. The summation is over the transient states. Thank you. May 3 '16 at 6:39
• In a realization, are the values of $T$ in the left hand side and the right hand side the same? In that case, let the value of $T$ be $a$. Then how a random value $\Phi(a)$ in the rhs can be the same with the $a$ in lhs? May 3 '16 at 6:52
• @Tayme: I'm afraid I don't understand that question. It sounds to me like a rephrasing of the original question, which I thought I'd addressed. Perhaps take as an example $f(X)=2X$ and $g(X)=X/2$, then $f(g(X))=X$. Do you see a problem in $X$ occurring on both sides of that equation? If not, what is it about the equation that you're asking about that makes it more problematic? May 3 '16 at 10:34

This is known as an absorbing Markov chain. Let $k$ be the absorbing state, that is, $\mathbb P(X_{n+1}=k\mid X_n=k)=1$. Then for each state $i\ne k$ we have the hitting time $$T_i := \inf\{n>0:X_n=k\mid X_0=i).$$

The $\Phi_k$ isn't standard notation to my knowledge, but it is true that $$T_i = \sum_{m=1}^{T_i} m\cdot\mathsf 1_{\{T_i=m\mid X_0=i\}},$$ since each term would be zero except for $m=T_i$, which would simply be $T_i$.

• I don't see how this is related to the question. Also it's confusing that you use $k$ in a different role than the question does; and $\mathbb P(T_i<\infty)=1$ isn't an additional assumption but a consequence of the fact that the absorbing state is recurrent. May 3 '16 at 6:12
• @joriki I think it is agreed that the statement was not correct as written, and that was my interpretation of what was intended. May 3 '16 at 6:21