1
$\begingroup$

Consider a Markov Chain $\{X_n\}$ on $S=\{0,1,\dots,d\}$ where $0,d$ are absorbing states and all other are transient. Also consider any transient state leads to every state. Let the transition probabilities be denoted by $p_{ij}$. Consider a transient state $\tilde{x}$. Define a new Markov Chain $\{\tilde{X}_n\}$ with transition probabilities $\tilde{p}_{ij}=p_{ij}$ for all $i\neq 0,d$ and define $\tilde{p}_{0\tilde{x}}=\tilde{p}_{d\tilde{x}}=1$. Show the new chain is irreducible, and hence has a unique stationary distribution. Also show that the expected absorption time in the first chain starting from $\tilde{x}$ is $(\tilde{\pi}_0+\tilde{\pi}_d)^{-1}-1$.

Now I have proved the irreducible part. Suppose we start from a transient state(in $X$). Say $i$, and we want to go to $j$, any other state. Then we know we have a path of nonzero probability (in the previous chain) that doesn't go to $0,d$ in between. Now if we consider this path in the new chain, it has the same transition probabilities, since in each step we start from a transient state. The only possible case when $0$ or $d$ may feature in the path is when the path ends there. Hence this path has positive probability even in the new chain. So we can go to any state from a transient state and if we start from $0,d$ we go to $\tilde{x}$ in the next step and from there we can reach any state. Now any finite state irreducible chain is positive recurrent hence there is a unique probability distribution. Now how to find the expected absorption time? Some help would be nice. Thanks a lot. Edit : I forgot to add a point, now I have edited it. Sorry for the confusion.

$\endgroup$
5
  • 1
    $\begingroup$ On the irreducibility: "Then we know we have a path of nonzero probability (in the previous chain) that doesn't go to $0,d$ in between." Perhaps I'm missing something but I see nothing in the question indicating that all the transient states intercommunicate in $\{X_n\}$ (and then $\{\tilde{X}_n\}$ is not necessarily irreducible). $\endgroup$
    – Mick A
    Feb 19, 2016 at 3:45
  • $\begingroup$ See any path from $i$ to $j$ cannot pass through $0$ or $d$ because they are absorbing states. And if they at all pass through them $j$ must be $0$ or $d$. $\endgroup$
    – shadow10
    Feb 19, 2016 at 5:17
  • 1
    $\begingroup$ But what if there is no path from $i$ to $j$ in either chain? $\endgroup$
    – Mick A
    Feb 19, 2016 at 5:51
  • 1
    $\begingroup$ Sorry I forgot to mention that every transient state leads to any state in $S$. So we always have a path between $i$ and $j$. $\endgroup$
    – shadow10
    Feb 19, 2016 at 16:23
  • $\begingroup$ Ok, thanks. That makes sense now. $\endgroup$
    – Mick A
    Feb 19, 2016 at 22:10

1 Answer 1

2
$\begingroup$

In chain $\tilde{X}$ we know that $\;\tilde{\pi}_0+\tilde{\pi}_d\;$ is the mean time spent in the set $A:=\{0,d\}$, and also that the mean recurrence time $\tilde{m}_A$ (i.e. starting in $A$, the mean number of steps to return to $A$) is its reciprocal:

$$\tilde{m}_A = \dfrac{1}{\tilde{\pi}_0+\tilde{\pi}_d}.$$

Every path starting in $A$ and returning to $A$ has its first step to state $\tilde{x}$. So the mean time to make the rest of the return trip from $\tilde{x}$ to $A$ equals $\tilde{m}_A - 1$.

The transition probabilities of each step in that path from $\tilde{x}$ to $A$ are the same in chains $\tilde{X}$ and $X$. Hence, the mean time to make the trip from $\tilde{x}$ to $A$ in $X$ is:

$$\tilde{m}_A - 1 = \dfrac{1}{\tilde{\pi}_0+\tilde{\pi}_d} - 1.$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .