Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

To find the stationary distribution of a Markov Chain, I believe I must solve for $\vec{s} = \langle s_0, s_1 \rangle$ in $\vec{s} = \vec{s}Q$, where $Q$ is the transition matrix.

$Q$, in my case, is

$$ \left( \begin{array}{cc} p & 1-p \\ 1-q & q \end{array} \right) $$

where $Q_{ij}$ is the probability of moving from state $i$ to state $j$ (row $i$, column $j$). When I solve for $s_0$ and $s_1$, however, I get

$$ s_0 = s_0 p + s_1 (1-q) \\ s_1 = s_0 (1 - p) + s_1 q $$

Subsequently,

$$ s_0 (1 - p) = s_1 (1 - q) \\ s_1 (1 - q) = s_0 (1 - p) $$

These two equations look identical. Does that mean there are an infinite number of stationary distributions for this Markov chain?

Thanks for helping a Markov Chain newb :)

share|improve this question
3  
no, it does not. Remember that you have an additional condition $s_0+s_1=1$. –  Artem Dec 8 '12 at 1:35
    
Ah, thank you. Why does that have to be a condition? –  David Faux Dec 8 '12 at 1:38
    
Because these are probabilities which have to sum to one. Or I don't understand your question? –  Artem Dec 8 '12 at 1:38
    
Ohhh... wait, why must the probabilities sum to 1? –  David Faux Dec 8 '12 at 1:41
    
Well, think what it would mean if probabilities added up to $3$ --- or to $-7$. –  Gerry Myerson Dec 8 '12 at 6:00
add comment

1 Answer 1

up vote 1 down vote accepted

As in the comments, the condition you are missing is that the probabilities must sum to 1 ($s_0+s_1=1$).

However, this shouldn't surprise you. There is always "one more equation than you need and one missing" when you write them out like this. That missing condition is the normalisation, that the probabilities must sum to 1. Why is there an extra equation?

It's a property of the $Q$ matrix. The $Q$ matrix has rank 1 less than its dimension (here rank 1) because the final column is redundant. If it were left blank you could 'fill in the gaps': $$\begin{pmatrix} p & *\\ 1-q & ** \end{pmatrix}.$$ You know that each row is a probability vector because it describes what can potentially happen. When in state $0$ the chain can remain in state $0$ or move to state $1$. As these are the only things that can happen the probabilities $p+*=1$ so $*=1-p$. Similarly $**=q$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.