Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a markov chain where the next state is related to the previous state by the following matrix: $$\begin{array}{c|ccc} & A & B & C\\ \hline A & p_1 & q_1 & r_1\\ B & p_2 & q_2 & r_2\\ C & p_3 & q_3 & r_3\\ \end{array}$$ Where the system of equations is given by

A$_{n+1} = p_1A_n + p_2B_n+p_3C_n$

B$_{n+1} = q_1A_n + q_2B_n+q_3C_n$

C$_{n+1} = r_1A_n + r_2B_n+r_3C_n$

How can the final relationships be found? (i.e. the limit of each function as n $\rightarrow \infty$, assuming A$_0 = B_0 = C_0$).

share|cite|improve this question
up vote 3 down vote accepted

If there is a limit, it will satisfy $$\eqalign{A&=p_1A+p_2B+p_3C\cr B&=q_1A+q_2B+q_3C\cr C&=r_1A+r_2B+r_3C\cr}$$ so it's just a matter of solving a system of three linear equations in three unknowns.

share|cite|improve this answer
The linear system you give is underdetermined, since all the columns of the transition matrix (had better) sum to one. To get a unique solution, you need to drop one of the equation and replace it with a normalization condition, e.g. $A + B + C = 1$. – Ilmari Karonen Mar 5 '12 at 2:36
Also, of course, even that doesn't guarantee that the solution is unique, nor does uniqueness of the solution guarantee that the chain actually converges. For example, the chain given by $p_1 = q_3 = r_2 = 1$ and $p_2 = p_3 = q_1 = q_2 = r_1 = r_3 = 0$ neither has a unique stationary distribution nor converges to any stationary distribution from most starting points. – Ilmari Karonen Mar 5 '12 at 2:40
Quite right. I didn't want to write a complete treatise on Markov chains, just enough to get OP started, especially since OP has put no effort into letting us know what level of response is required. – Gerry Myerson Mar 5 '12 at 2:55
But you can't solve the system of equations, because you have six unknowns, since we don't know A, B, C or A$_{n+1}$, B$_{n+1}$, or C$_{n+1}$ – SSumner Mar 5 '12 at 15:27
The ones with the subscripts - where do you see those in the equations I wrote? The only unknowns are $A$, $B$, and $C$. – Gerry Myerson Mar 6 '12 at 0:24

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.