Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have drawn a certain Markov chain with a weird transition matrix. Here's the drawing:

And here's the transition matrix:

My problem is that I don't quite know how to calculate the steady state probabilities of this chain, if it exists. I need a bit of help, because just getting the $\pi$'s, they don't sum to 1 for me, which is weird.

I've assembled this Markov chain using some data and conditions given to me and, right now, I would insist that it is the inevitable result of a very long process of solving. But now I hit a brick wall and can't continue. Please help?


Here's my solution to the problem, using the ordinary ways of calculating steady states.


As you can see, it's a little bit problematic. The probabilities don't add up to 1. I've turned it upside down several times. Is it really just something about my arithmetic, or is this simply a weird Markov chain?

share|cite|improve this question
picture is gone – Double AA Apr 26 '15 at 21:15
up vote 1 down vote accepted

There are two equations here: $\pi P=\pi$ and $\sum_i \pi_i=1$. So you will get eight equations to solve for seven variables. You need to omit one of the equations you get from $\pi P=\pi$ and solve the linear equations.

$$P=\left[\begin{array}{ccccccc}l&m&n&0&0&0&0\\o&p&0&0&0&0&0\\ o&p&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\end{array}\right]$$

These are the linear equations you need to solve:

$$l\pi_0+o(\pi_1+\pi_2)+\pi_3+\pi_4+\pi_5+\pi_6=\pi_0\\ m\pi_0+p(\pi_1+\pi_2)=\pi_1\\ n\pi_0=\pi_2\\ \pi_0+\pi_1+\pi_2+\pi_3+\pi_4+\pi_5+\pi_6=1$$

share|cite|improve this answer
Hi Shyam. You're right. I did those calculations and will post the resulting image on the OP right now. I basically have a problem because the pi's don't add up to 1. – markovchain Mar 3 '13 at 21:33
@user64834: The $\pi$s won't naturally add up to 1 - you have to specify that as a constraint while solving. – Bravo Mar 3 '13 at 21:39
Thanks. I got the answer. :) It's 5am, must be the lack of sleep. – markovchain Mar 3 '13 at 21:45

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.