# Probability distribution of markov chain

I have a Markov chain with state space $E = \{1,2,3,4,5\}$ and transition matrix below:

$$\begin{bmatrix} 1/2 & 0 & 1/2 & 0 & 0 \\ 1/3 & 2/3 & 0 & 0 & 0 \\ 0 & 1/4 & 1/4 & 1/4 & 1/4 \\ 0 & 0 & 0 & 3/4 & 1/4 \\ 0 & 0 & 0 & 1/5 & 4/5 \end{bmatrix}$$

Given the initial distribution $\pi = (1/2,0,0,1/2,0)$, how would I calculate $\mathbb{P}(X_2 = 4)$? Do I have to multiply $\pi$ by the transition matrix etc.?

-

If "initial distribution" means the distribution of $X_1$, then the distribution of $X_2$ is $\pi M$, where $M$ is the transition matrix above.

If "initial distribution" means the distribution of $X_0$, then the distribution of $X_2$ is $\pi M^2$.

-
Thanks Michael! I think the question means the "initial distribution" means the distribution of $X_0$. Just one quick question, I cant seem to multiply $M^2 \pi$, because the dimensions (row and columns) are not correct, but I can multiply $\pi M^2$, and I get the correct answer $(59/160)$. So the distribution $M^2 \pi$ or $\pi M^2$? –  Henry Apr 10 '12 at 23:41
It should be $\pi M^2$. This is a "right stochastic matrix" (rows sum up to 1), which means it's on the "right" of a matrix multiplication. It is my understanding that transition matrices are by convention situated on the right. I'm not sure why this is so; maybe someone else can enlighten us. –  Yang Apr 11 '12 at 0:01
Yes, please, can someone enlighten us! @X.Yang, so if I have a left stochastic matrix, would it be on the "left" of a matrix multiplication? Or right? –  Henry Apr 11 '12 at 0:08
On the left - that's where the name comes from. A left stochastic matrix is defined as one where all the columns sum up to 1. This is essential because, when you end up doing the matrix multiplication, you'll see that each state has a total probability 1 of going to any other state - which only makes sense. Same thing for right stochastic matrices. –  Yang Apr 11 '12 at 0:15
Thanks, what do you mean then when you said that by convention the transition matrices are situation on the right? –  Henry Apr 11 '12 at 0:20