I need to find an expression for the first state in an MC with transition matrix $P$ with tridiagonal entries. The state space is $U={1,2,..n}$ with the last state being absorbing. Expressions for limiting distribution for the other states are given by $$ \pi_1 = r_{11} \pi_1 + q_{21} \pi_2 \\ \pi_j = r_{jj} \pi_j + q_{j+1, j} \pi_{j+1} + p_{j-1,j} \pi_{j-1} $$

I transform this matrix by adding probability $p_{n,1}=1$ to remove the absorbing state. By solving the recurent equation I get the expression for $\pi_j, j>1$:

$ \pi_j = \frac{p_{12} p_{23} ... p_{j-1,j}}{q_{21}q_{23}...q_{j,j-1}} \pi_1 $

So my question is, how to find $\pi_1$? It's not a constant value, my impression it is also expressed in the form $\frac{p}{q}$, but I'm not too sure.

  • $\begingroup$ What makes you think that replacing $r_{n,n}=1$ (absorption at $n$) by $p_{n,1}=1$ (full probability transition from $n$ to $1$) yields a Markov chain whose stationary distribution is related in one way or another to the quasi-stationary distribution of the original Markov chain? $\endgroup$ – Did Feb 14 '12 at 8:34
  • $\begingroup$ You are looking for the solution the a linear equation: if $\pi^*$ solves it then $k\pi^*$ solves it for all $k\in\mathbb R$. Since you are looking for the probability distribution, don't forget to apply the condition $\sum\limits_{j=1}^n \pi_j = 1$. $\endgroup$ – Ilya Feb 14 '12 at 9:04
  • $\begingroup$ @Ilya:of course! Spasibo) $\endgroup$ – sigma.z.1980 Feb 14 '12 at 23:54
  • $\begingroup$ @Sigma Ne za 4to $\endgroup$ – Ilya Feb 15 '12 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.