Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider the $3\times3$ transition matrix

$$ \begin{array}{c|ccc} &A &B &C \\ \hline A &0.2 & 0.3 &0.5\\ B &0.3 & 0.5 &0.2\\ C &0.3 & 0.3 & 0.4 \end{array}. $$ Given that the chain starts in state $B$ what is the probability that after three transitions it is back in state $B$ having visited $A$ and $C$?

I have a few exam paper questions similar to this but I can only find one sort of formula related to this in my notes and it's not very clear and no examples. I was wondering if someone could give me a formula for these types of problems and show me how to apply it to this question and how to use it for others?

Many thanks in advance. :)

share|cite|improve this question
up vote 3 down vote accepted

$$B\Rightarrow C\Rightarrow A\Rightarrow B$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=P(x_0=B)\ P(x_1=C|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=A|x_1=C)\ P(x_3=B|x_2=A)$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=1\times 0.2\times 0.3\times 0.3=0.018$$ or $$B\Rightarrow A\Rightarrow C\Rightarrow B$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=P(x_0=B)\ P(x_1=A|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=C|x_1=A)\ P(x_3=B|x_2=C)$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=1\times 0.3\times 0.5\times 0.3=0.045$$ and in total $$P=0.063$$ -----------Transition Matrix------------ \begin{array}{c|ccc} &A &B &C \\ \hline A &P(A|A) &P(B|A) &P(C|A)\\ B &P(A|B) &P(B|B) &P(C|B)\\ C &P(A|C) &P(B|C) &P(C|C) \end{array} which means if you select (2,1) it means that you look up for the probability of transition from B to A. Rows are for current state and columns for next state.

share|cite|improve this answer
So say a chain starts in state C what is the probability that after two transitions it is in A? – Swayy Mar 30 '13 at 1:26
I meant to add: $C \rightarrow B \rightarrow A$ or $C \rightarrow A \rightarrow A$ or $C \rightarrow C \rightarrow A$ So taking the first one: $P(x_0 = C)P(x_1 = B |x_0 = C)P(x_2 = A | x_1 = B)P(x_0=C, x_1 = B, x_3 = A)$ I guess what I'm asking is where are the numbers coming from. I'm a little confused. Thanks :) – Swayy Mar 30 '13 at 1:32
@Swayy check for edit. – Occupy Gezi Mar 30 '13 at 7:34
That's perfect! Thanks so much! :) – Swayy Mar 30 '13 at 12:07

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.