Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have drawn a certain Markov chain with a weird transition matrix. Here's the drawing:

https://dl.dropbox.com/u/106018191/markovpolicy.png

And here's the transition matrix:

https://dl.dropbox.com/u/106018191/mathstackex.png

My problem is that I don't quite know how to calculate the steady state probabilities of this chain, if it exists. I need a bit of help, because just getting the $\pi$'s, they don't sum to 1 for me, which is weird.

I've assembled this Markov chain using some data and conditions given to me and, right now, I would insist that it is the inevitable result of a very long process of solving. But now I hit a brick wall and can't continue. Please help?

Edit:

Here's my solution to the problem, using the ordinary ways of calculating steady states.

solution

As you can see, it's a little bit problematic. The probabilities don't add up to 1. I've turned it upside down several times. Is it really just something about my arithmetic, or is this simply a weird Markov chain?

share|improve this question

1 Answer 1

up vote 0 down vote accepted

There are two equations here: $\pi P=\pi$ and $\sum_i \pi_i=1$. So you will get eight equations to solve for seven variables. You need to omit one of the equations you get from $\pi P=\pi$ and solve the linear equations.

$$P=\left[\begin{array}{ccccccc}l&m&n&0&0&0&0\\o&p&0&0&0&0&0\\ o&p&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\end{array}\right]$$

These are the linear equations you need to solve:

$$l\pi_0+o(\pi_1+\pi_2)+\pi_3+\pi_4+\pi_5+\pi_6=\pi_0\\ m\pi_0+p(\pi_1+\pi_2)=\pi_1\\ n\pi_0=\pi_2\\ \pi_0+\pi_1+\pi_2+\pi_3+\pi_4+\pi_5+\pi_6=1$$

share|improve this answer
    
Hi Shyam. You're right. I did those calculations and will post the resulting image on the OP right now. I basically have a problem because the pi's don't add up to 1. –  markovchain Mar 3 '13 at 21:33
    
@user64834: The $\pi$s won't naturally add up to 1 - you have to specify that as a constraint while solving. –  Bravo Mar 3 '13 at 21:39
    
Thanks. I got the answer. :) It's 5am, must be the lack of sleep. –  markovchain Mar 3 '13 at 21:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.