Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a Markov chain with state space $E = \{1,2,3,4,5\}$ and transition matrix below:

$$ \begin{bmatrix} 1/2 & 0 & 1/2 & 0 & 0 \\ 1/3 & 2/3 & 0 & 0 & 0 \\ 0 & 1/4 & 1/4 & 1/4 & 1/4 \\ 0 & 0 & 0 & 3/4 & 1/4 \\ 0 & 0 & 0 & 1/5 & 4/5 \end{bmatrix} $$

Given the initial distribution $\pi = (1/2,0,0,1/2,0)$, how would I calculate $\mathbb{P}(X_2 = 4)$? Do I have to multiply $\pi$ by the transition matrix etc.?

share|cite|improve this question
up vote 1 down vote accepted

If "initial distribution" means the distribution of $X_1$, then the distribution of $X_2$ is $\pi M$, where $M$ is the transition matrix above.

If "initial distribution" means the distribution of $X_0$, then the distribution of $X_2$ is $\pi M^2$.

share|cite|improve this answer
Thanks Michael! I think the question means the "initial distribution" means the distribution of $X_0$. Just one quick question, I cant seem to multiply $M^2 \pi$, because the dimensions (row and columns) are not correct, but I can multiply $\pi M^2$, and I get the correct answer $(59/160)$. So the distribution $M^2 \pi$ or $\pi M^2$? – Henry Apr 10 '12 at 23:41
It should be $\pi M^2$. This is a "right stochastic matrix" (rows sum up to 1), which means it's on the "right" of a matrix multiplication. It is my understanding that transition matrices are by convention situated on the right. I'm not sure why this is so; maybe someone else can enlighten us. – Yang Apr 11 '12 at 0:01
Yes, please, can someone enlighten us! @X.Yang, so if I have a left stochastic matrix, would it be on the "left" of a matrix multiplication? Or right? – Henry Apr 11 '12 at 0:08
On the left - that's where the name comes from. A left stochastic matrix is defined as one where all the columns sum up to 1. This is essential because, when you end up doing the matrix multiplication, you'll see that each state has a total probability 1 of going to any other state - which only makes sense. Same thing for right stochastic matrices. – Yang Apr 11 '12 at 0:15
Thanks, what do you mean then when you said that by convention the transition matrices are situation on the right? – Henry Apr 11 '12 at 0:20

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.