Two questions:

Given the transition matrix:

$\begin{vmatrix} \ 0.4 & 0.4 & 0.2 \\ \ 0.5 & 0.3 & 0.2 \\ \ 0.1 & 0.5 & 0.4 \end{vmatrix}$

I would like to know HOW to find the steady-state of this.

I've tried to do that trying to solve the system by hand:

\begin{aligned} \ a = 0.4a+0.5b+0.1c \\ \ b = 0.4a+0.3b+0.5c \\ \ c = 0.2a+0.2b+0.4c \\ \ 1 = a+b+c \\ \end{aligned}

Question 1: Is it possible to solve this system by hand? What is the best way to do that? If it not possible to solve, how do i see this impossibility?

Question 2: Wolfram said to me that this system is impossible (please, answer the first question (:). What this fact means to this Markov chain?

-
Typo in your equation for b? 0.5a? – Axel Kemper Apr 7 '13 at 20:59
Yes. I am sorry. Now I think it is right. – Richard Apr 7 '13 at 21:18
It's impossible that it's impossible. Every finite state space Markov chain has a steady state probability vector. – Byron Schmuland Apr 7 '13 at 21:31

In the steady-state, the vector of probabilities multiplied by the transition matrix has to result in the vector of probabilities. Therefore, you have to look for the Eigenvector of the matrix associated to Eigenvalue 1.

A brute-force solution is to use Excel or pen and paper and multiply some assumed initial vector of probabilities with the matrix. If you repeat that a number of times with the resulting vector as new input, you end up at:

a = 0.363636
b = 0.386363
c = 0.25


A related question/discussion is here.

-
Or as fractions $(a,b,c)=(4/11,17/44,1/4)$. – Byron Schmuland Apr 7 '13 at 22:06
Please, show how to find these values without using Excel or brute-force. – Richard Apr 8 '13 at 2:10
There is an online calculator available with some explanations: math.plussed.net/markov – Axel Kemper Apr 8 '13 at 6:33