0
$\begingroup$

Let's think about a discrete time Markov chain $X_t$ with only one recurrent state. Let $T$ be the random variable that is the number of steps taken from a given state $i$ to the recurrent state (ie. the first passage time to the recurrent state) and $\Phi_k(n) = \sum_{t=0}^n I(X_t = k | X_0 = i)$ be the random variable of occupation time of state $k$ during $n$ steps, where $I$ is the indicator function. Today, in a class on stochastic process, I learned a relation between them : $T=\sum_k \Phi_k(T)$. I'm very confused with this. Is it really legal? How the same random variable $T$ can be appeared in two places in an equation? I have no idea how to interpret the relation. Could someone explain it in plain English?

$\endgroup$
0
$\begingroup$

Yes, this is legal. The right-hand side is a function of a random variable, which again yields a random variable, and the equation states that the random variables on its two sides are equal, which is also a meaningful statement.

The statement isn't quite correct, however; the left-hand side should be $T+1$, or the summation on the right-hand side should be restricted to the non-recurrent states. The process was in exactly one state at any given time, so the sum on the right-hand side contains exactly $T+1$ ones.

$\endgroup$
3
  • $\begingroup$ You are right. The summation is over the transient states. Thank you. $\endgroup$
    – Tayme
    May 3 '16 at 6:39
  • $\begingroup$ In a realization, are the values of $T$ in the left hand side and the right hand side the same? In that case, let the value of $T$ be $a$. Then how a random value $\Phi(a)$ in the rhs can be the same with the $a$ in lhs? $\endgroup$
    – Tayme
    May 3 '16 at 6:52
  • $\begingroup$ @Tayme: I'm afraid I don't understand that question. It sounds to me like a rephrasing of the original question, which I thought I'd addressed. Perhaps take as an example $f(X)=2X$ and $g(X)=X/2$, then $f(g(X))=X$. Do you see a problem in $X$ occurring on both sides of that equation? If not, what is it about the equation that you're asking about that makes it more problematic? $\endgroup$
    – joriki
    May 3 '16 at 10:34
0
$\begingroup$

This is known as an absorbing Markov chain. Let $k$ be the absorbing state, that is, $\mathbb P(X_{n+1}=k\mid X_n=k)=1$. Then for each state $i\ne k$ we have the hitting time $$T_i := \inf\{n>0:X_n=k\mid X_0=i).$$

The $\Phi_k$ isn't standard notation to my knowledge, but it is true that $$T_i = \sum_{m=1}^{T_i} m\cdot\mathsf 1_{\{T_i=m\mid X_0=i\}},$$ since each term would be zero except for $m=T_i$, which would simply be $T_i$.

$\endgroup$
2
  • $\begingroup$ I don't see how this is related to the question. Also it's confusing that you use $k$ in a different role than the question does; and $\mathbb P(T_i<\infty)=1$ isn't an additional assumption but a consequence of the fact that the absorbing state is recurrent. $\endgroup$
    – joriki
    May 3 '16 at 6:12
  • $\begingroup$ @joriki I think it is agreed that the statement was not correct as written, and that was my interpretation of what was intended. $\endgroup$
    – Math1000
    May 3 '16 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.