Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

$P=\left(\matrix{0.7 \ \ \ \ 0.3\\0.6 \ \ \ \ 0.4}\right)$ be the transition probability matrix of finite markov chain wtih 2 states, $1,2$ and what is the probability that in long run, $P(X_n=1)$?

I know how to find stationary distribution but the solution suggest that it is equal to $\frac{0.6}{0.6+0.3}$ without using any stationary distribution and i don't quite get how it gets that.

share|cite|improve this question
Now that's rather original -- usually we have the problem that the title basically says nothing more than "problem" or "question" -- you've written a title in which the only nouns are "solution" and "answer" :-) Please choose a more specific title that summarizes the content of the question. – joriki Dec 16 '12 at 8:46
up vote 2 down vote accepted

You could try to show the following:

The stationary distribution of the Markov chain with transition matrix $\begin{pmatrix}1-a & a\\ b & 1-b\end{pmatrix}$ is proportional to $\begin{pmatrix}b\\ a\end{pmatrix}$, for every $a$ and $b$ in $(0,1)$.

Of course, such a characterization is specific to the two-states case.

share|cite|improve this answer
I see. As from the computation of the stationary distribution, we can get that $a\pi_1 - b\pi_2=0$ so we have $\frac{\pi_1}{\pi_2}=\frac{b}{a}$. – Mathematics Dec 16 '12 at 9:09

The markov chain is irreducible and finite state. Additionally it is aperiodic. Thus not only is it positive recurrent but it has a unique steady state distribution.

share|cite|improve this answer
Not sure this answers the question. – Did Dec 16 '12 at 9:12
I just wanted to fill in the gaps. He already knows how to solve it. – Gautam Shenoy Dec 16 '12 at 12:09

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.