Markov Chain Exercise.

Question

A Markov chain has the transition probability matrix

$$P= \begin{bmatrix} 0.7 & 0.2 & 0.1 \\ 0 & 0.6 & 0.4 \\ 0.5 & 0 & 0.5 \\ \end{bmatrix}. $$

Let us number the states $0,1,2$.

Now I have to determine (a) the conditional probability $Pr{\{X_2=1,X_3=1|X_1=0\}}$

(b) If it is known that the process starts in state $X_0=1$, what is the probability $Pr{\{X_0=1,X_1=0,X_2=2\}}$ ?

My Attempt:

(a) $$Pr{\{X_2=1,X_3=1|X_1=0\}}=Pr{\{X_2=1|X_1=0\}}Pr{\{X_3=1|X_1=0\}}=(0.2)Pr{\{X_3=1|X_1=0\}},$$

for calculating $Pr{\{X_3=1|X_1=0\}}$, I need two step transition probability matrix :

$$P^2= \begin{bmatrix} 0.54 & 0.26 & 0.2 \\ 0.2 & 0.36 & 0.44 \\ 0.6 & 0.1 & 0.3 \\ \end{bmatrix}. $$

Hence $$Pr{\{X_2=1,X_3=1|X_1=0\}}=(0.2)Pr{\{X_3=1|X_1=0\}}=(0.2)(0.26)=0.052,$$ which doesn't match with the result. Where am I doing mistake ?

(b)$$Pr{\{X_0=1,X_1=0,X_2=2\}}=Pr{\{X_0=1\}}P_{1,0}P_{0,2},$$ where $P_{0,2}$ denotes transition from state $0$ to state $2$ in one step.

But my question is the problem hasn't specified the initial distribution, so I am not getting the value of $Pr{\{X_0=1\}}$, consequently not getting $Pr{\{X_0=1,X_1=0,X_2=2\}}$. But the problem has an answer. How can I proceed ?

Answer 1

(a) Since the process if Markov, we can write

$$ P(X_2 = 1, X_3 = 1 \mid X_1 = 0) = P(X_2 = 1 \mid X_1 = 0) \cdot P(X_3 = 1 \mid X_2 = 1) $$

and you only need the transition matrix itself to answer.

(b) If you're given that $X_0 = 1$, then (as Antitheos also points out) $P(X_0 = 1) = 1$, and you can write

\begin{align} P(X_0 = 1, X_1 = 0, X_2 = 2) & = P(X_1 = 0, X_2 = 2 \mid X_0 = 1) \\ & = P(X_1 = 0 \mid X_0 = 1) \cdot P(X_2 = 2 \mid X_1 = 0) \end{align}

and you can proceed as before.

Answer 2

a) First, by the definition of the term $$P(X_2=1,X_3=1 |X_1=0) = P(X_3=1|X_2=1,X_1=0)\cdot P( X_2=1|X_1=0)$$ and by Markoc chain definition $$P(X_3=1|X_2=1,X_1=0)\cdot P( X_2=1|X_1=0) =P(X_3=1|X_2=1)\cdot P( X_2=1|X_1=0) =p_{1,1}\cdot p_{1,0} $$

b) Read the problem again:

it is known that the process starts in state $X_0=1$

This means $$P(X_0=1) =1.$$