1
$\begingroup$

I am studying stochastic processes and have stumbled on a result that is puzzling me. I have searched elsewhere for an answer without luck so hoping some proper mathematicians here can explain the result for me.

Given a two-state Markov process with probability transition matrix $$ \begin{array}{c|c} &\begin{matrix}0&1\end{matrix}\\ \hline \begin{matrix}0\\ 1\end{matrix} &\pmatrix{a&b\\ c&d} \end{array} $$

I have found that the simplest way to calculate its steady-state probability distribution is :

state 0: $c \over {b + c}$

state 1: $b \over {b + c}$

This result holds for all examples I have tried, but I have been unable to explain it from theory, so cannot prove it. My questions are:

  1. what is the theoretical explanation for this result?
  2. does it extend to any $n\times n$ transition matrix?
| cite | improve this question | | | | |
$\endgroup$
1
$\begingroup$

This is specific to two-state chains. Note that $a=1-b$ and $d=1-c$ for the matrix to be a probability transition matrix hence $(b,c)$ determines entirely the probability transition matrix and the distribution $(\pi_0,\pi_1)$ is stationary if and only if $$ \pi_0=a\pi_0+c\pi_1,\qquad\pi_1=b\pi_0+d\pi_1. $$ This system is equivalent to the single equation $$ b\pi_0=c\pi_1. $$ The normalization $\pi_0+\pi_1=1$ yields the values in the question.

| cite | improve this answer | | | | |
$\endgroup$
  • $\begingroup$ Clear, concise and helpful answer, many thanks. $\endgroup$ – vince Oct 11 '13 at 15:59
  • $\begingroup$ You are welcome. $\endgroup$ – Did Oct 11 '13 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.