I have problem with calculating the probability of Markov Chain with 3 states S = {0,1,2}.

I need to calculate $P(X_1=1,X_2=1|X_0=2)$.

In the answers to my workbook I am given solution:

$P(X_1=1,X_2=1|X_0=2) = P(X_2=1|X_1=1,X_0=2)P(X_1=1|X_0=2)P(X_2=1|X_1=1)P(X_1=1|X_0=2),$

but I have trouble understanding what happened in this step (I guess total probability is used, but I don't really get how).

There is also a transition matrix give and starting distribution, but I'm not sure if they are needed here.

Thanks a lot for help.


There should be another equal sign; the solution should look like this:

$\begin{align}P(X_1=1,X_2=1|X_0=2) &= P(X_2=1|X_1=1,X_0=2)P(X_1=1|X_0=2)\\ &=P(X_2=1|X_1=1)P(X_1=1|X_0=2) \end{align}$

where the first equality uses Bayes rule/the definition of conditional probability, applied partially; compare this with $P(X_2,X_1)=P(X_2|X_1)P(X_1)$

(or multiply both sides of $P(X_1=1,X_2=1|X_0=2) = P(X_2=1|X_1=1,X_0=2)P(X_1=1|X_0=2)$ by $P(X_0=2)$ by zero to check that the equality is true.)

while the second equality uses the Markov property.

  • $\begingroup$ How should I understand this first statement: $P(X_1=1,X_2=1|X_0=2) ?$ Does it mean Prob of (A and B) under condition C or is it Prob of A and (B under condition C)? $\endgroup$ – Przemysław Robert Wilk May 26 '15 at 21:40
  • $\begingroup$ That refers to the probability of A and B, conditioned on C. In this context, you are at state 2 at time 0, and are interested in the next two states being 1 at times 1 and 2 respectively. (On the other hand, you could think of $P(X_2=1 | X_1=1, X_0=2) as: you are currently at state 1 at time 1, and you have the historical information that you were in state 2 at time 0; you want the probability of moving to state 1 at time 2.) $\endgroup$ – Ken Wei May 26 '15 at 21:54
  • $\begingroup$ Thank's a lot Ken, this explanation really made me understand! $\endgroup$ – Przemysław Robert Wilk May 26 '15 at 22:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.