Consider the $3\times3$ transition matrix

$$ \begin{array}{c|ccc} &A &B &C \\ \hline A &0.2 & 0.3 &0.5\\ B &0.3 & 0.5 &0.2\\ C &0.3 & 0.3 & 0.4 \end{array}. $$ Given that the chain starts in state $B$ what is the probability that after three transitions it is back in state $B$ having visited $A$ and $C$?

I have a few exam paper questions similar to this but I can only find one sort of formula related to this in my notes and it's not very clear and no examples. I was wondering if someone could give me a formula for these types of problems and show me how to apply it to this question and how to use it for others?

Many thanks in advance. :)


$$B\Rightarrow C\Rightarrow A\Rightarrow B$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=P(x_0=B)\ P(x_1=C|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=A|x_1=C)\ P(x_3=B|x_2=A)$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=1\times 0.2\times 0.3\times 0.3=0.018$$ or $$B\Rightarrow A\Rightarrow C\Rightarrow B$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=P(x_0=B)\ P(x_1=A|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=C|x_1=A)\ P(x_3=B|x_2=C)$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=1\times 0.3\times 0.5\times 0.3=0.045$$ and in total $$P=0.063$$ -----------Transition Matrix------------ \begin{array}{c|ccc} &A &B &C \\ \hline A &P(A|A) &P(B|A) &P(C|A)\\ B &P(A|B) &P(B|B) &P(C|B)\\ C &P(A|C) &P(B|C) &P(C|C) \end{array} which means if you select (2,1) it means that you look up for the probability of transition from B to A. Rows are for current state and columns for next state.

  • $\begingroup$ So say a chain starts in state C what is the probability that after two transitions it is in A? $\endgroup$ – Swayy Mar 30 '13 at 1:26
  • $\begingroup$ I meant to add: $C \rightarrow B \rightarrow A$ or $C \rightarrow A \rightarrow A$ or $C \rightarrow C \rightarrow A$ So taking the first one: $P(x_0 = C)P(x_1 = B |x_0 = C)P(x_2 = A | x_1 = B)P(x_0=C, x_1 = B, x_3 = A)$ I guess what I'm asking is where are the numbers coming from. I'm a little confused. Thanks :) $\endgroup$ – Swayy Mar 30 '13 at 1:32
  • 1
    $\begingroup$ @Swayy check for edit. $\endgroup$ – AnilB Mar 30 '13 at 7:34
  • $\begingroup$ That's perfect! Thanks so much! :) $\endgroup$ – Swayy Mar 30 '13 at 12:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.