# Markov chains - transition matrix - probabilty formula and application help?

Consider the $3\times3$ transition matrix

$$\begin{array}{c|ccc} &A &B &C \\ \hline A &0.2 & 0.3 &0.5\\ B &0.3 & 0.5 &0.2\\ C &0.3 & 0.3 & 0.4 \end{array}.$$ Given that the chain starts in state $B$ what is the probability that after three transitions it is back in state $B$ having visited $A$ and $C$?

I have a few exam paper questions similar to this but I can only find one sort of formula related to this in my notes and it's not very clear and no examples. I was wondering if someone could give me a formula for these types of problems and show me how to apply it to this question and how to use it for others?

$$B\Rightarrow C\Rightarrow A\Rightarrow B$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=P(x_0=B)\ P(x_1=C|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=A|x_1=C)\ P(x_3=B|x_2=A)$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=1\times 0.2\times 0.3\times 0.3=0.018$$ or $$B\Rightarrow A\Rightarrow C\Rightarrow B$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=P(x_0=B)\ P(x_1=A|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=C|x_1=A)\ P(x_3=B|x_2=C)$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=1\times 0.3\times 0.5\times 0.3=0.045$$ and in total $$P=0.063$$ -----------Transition Matrix------------ \begin{array}{c|ccc} &A &B &C \\ \hline A &P(A|A) &P(B|A) &P(C|A)\\ B &P(A|B) &P(B|B) &P(C|B)\\ C &P(A|C) &P(B|C) &P(C|C) \end{array} which means if you select (2,1) it means that you look up for the probability of transition from B to A. Rows are for current state and columns for next state.
• I meant to add: $C \rightarrow B \rightarrow A$ or $C \rightarrow A \rightarrow A$ or $C \rightarrow C \rightarrow A$ So taking the first one: $P(x_0 = C)P(x_1 = B |x_0 = C)P(x_2 = A | x_1 = B)P(x_0=C, x_1 = B, x_3 = A)$ I guess what I'm asking is where are the numbers coming from. I'm a little confused. Thanks :) Mar 30 '13 at 1:32