3
$\begingroup$

I have drawn a certain Markov chain with a weird transition matrix. Here's the drawing:

https://dl.dropbox.com/u/106018191/markovpolicy.png

And here's the transition matrix:

https://dl.dropbox.com/u/106018191/mathstackex.png

My problem is that I don't quite know how to calculate the steady state probabilities of this chain, if it exists. I need a bit of help, because just getting the $\pi$'s, they don't sum to 1 for me, which is weird.

I've assembled this Markov chain using some data and conditions given to me and, right now, I would insist that it is the inevitable result of a very long process of solving. But now I hit a brick wall and can't continue. Please help?

Edit:

Here's my solution to the problem, using the ordinary ways of calculating steady states.

solution

As you can see, it's a little bit problematic. The probabilities don't add up to 1. I've turned it upside down several times. Is it really just something about my arithmetic, or is this simply a weird Markov chain?

$\endgroup$
2
  • $\begingroup$ picture is gone $\endgroup$
    – Double AA
    Apr 26, 2015 at 21:15
  • $\begingroup$ Wow, great username, OP $\endgroup$ Apr 22, 2021 at 4:23

1 Answer 1

1
$\begingroup$

There are two equations here: $\pi P=\pi$ and $\sum_i \pi_i=1$. So you will get eight equations to solve for seven variables. You need to omit one of the equations you get from $\pi P=\pi$ and solve the linear equations.

$$P=\left[\begin{array}{ccccccc}l&m&n&0&0&0&0\\o&p&0&0&0&0&0\\ o&p&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\end{array}\right]$$

These are the linear equations you need to solve:

$$l\pi_0+o(\pi_1+\pi_2)+\pi_3+\pi_4+\pi_5+\pi_6=\pi_0\\ m\pi_0+p(\pi_1+\pi_2)=\pi_1\\ n\pi_0=\pi_2\\ \pi_0+\pi_1+\pi_2+\pi_3+\pi_4+\pi_5+\pi_6=1$$

$\endgroup$
3
  • $\begingroup$ Hi Shyam. You're right. I did those calculations and will post the resulting image on the OP right now. I basically have a problem because the pi's don't add up to 1. $\endgroup$ Mar 3, 2013 at 21:33
  • $\begingroup$ @user64834: The $\pi$s won't naturally add up to 1 - you have to specify that as a constraint while solving. $\endgroup$
    – Bravo
    Mar 3, 2013 at 21:39
  • $\begingroup$ Thanks. I got the answer. :) It's 5am, must be the lack of sleep. $\endgroup$ Mar 3, 2013 at 21:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .