1
$\begingroup$

A Markov chain has the transition probability matrix

$$P= \begin{bmatrix} 0.7 & 0.2 & 0.1 \\ 0 & 0.6 & 0.4 \\ 0.5 & 0 & 0.5 \\ \end{bmatrix}. $$

Let us number the states $0,1,2$.

Now I have to determine (a) the conditional probability $Pr{\{X_2=1,X_3=1|X_1=0\}}$

(b) If it is known that the process starts in state $X_0=1$, what is the probability $Pr{\{X_0=1,X_1=0,X_2=2\}}$ ?

My Attempt:

(a) $$Pr{\{X_2=1,X_3=1|X_1=0\}}=Pr{\{X_2=1|X_1=0\}}Pr{\{X_3=1|X_1=0\}}=(0.2)Pr{\{X_3=1|X_1=0\}},$$

for calculating $Pr{\{X_3=1|X_1=0\}}$, I need two step transition probability matrix :

$$P^2= \begin{bmatrix} 0.54 & 0.26 & 0.2 \\ 0.2 & 0.36 & 0.44 \\ 0.6 & 0.1 & 0.3 \\ \end{bmatrix}. $$

Hence $$Pr{\{X_2=1,X_3=1|X_1=0\}}=(0.2)Pr{\{X_3=1|X_1=0\}}=(0.2)(0.26)=0.052,$$ which doesn't match with the result. Where am I doing mistake ?

(b)$$Pr{\{X_0=1,X_1=0,X_2=2\}}=Pr{\{X_0=1\}}P_{1,0}P_{0,2},$$ where $P_{0,2}$ denotes transition from state $0$ to state $2$ in one step.

But my question is the problem hasn't specified the initial distribution, so I am not getting the value of $Pr{\{X_0=1\}}$, consequently not getting $Pr{\{X_0=1,X_1=0,X_2=2\}}$. But the problem has an answer. How can I proceed ?

$\endgroup$
  • $\begingroup$ First problem: $$Pr{\{X_2=1,X_3=1|X_1=0\}}\ne Pr{\{X_2=1|X_1=0\}}Pr{\{X_3=1|X_1=0\}}$$ $\endgroup$ – Did Oct 1 '15 at 20:26
  • $\begingroup$ Six minutes. $ $ $\endgroup$ – Did Oct 1 '15 at 20:35
  • $\begingroup$ @Did what is six minutes ? $\endgroup$ – ABC Oct 1 '15 at 20:36
2
$\begingroup$

(a) Since the process if Markov, we can write

$$ P(X_2 = 1, X_3 = 1 \mid X_1 = 0) = P(X_2 = 1 \mid X_1 = 0) \cdot P(X_3 = 1 \mid X_2 = 1) $$

and you only need the transition matrix itself to answer.

(b) If you're given that $X_0 = 1$, then (as Antitheos also points out) $P(X_0 = 1) = 1$, and you can write

\begin{align} P(X_0 = 1, X_1 = 0, X_2 = 2) & = P(X_1 = 0, X_2 = 2 \mid X_0 = 1) \\ & = P(X_1 = 0 \mid X_0 = 1) \cdot P(X_2 = 2 \mid X_1 = 0) \end{align}

and you can proceed as before.

$\endgroup$
-1
$\begingroup$

a) First, by the definition of the term $$P(X_2=1,X_3=1 |X_1=0) = P(X_3=1|X_2=1,X_1=0)\cdot P( X_2=1|X_1=0)$$ and by Markoc chain definition $$P(X_3=1|X_2=1,X_1=0)\cdot P( X_2=1|X_1=0) =P(X_3=1|X_2=1)\cdot P( X_2=1|X_1=0) =p_{1,1}\cdot p_{1,0} $$

b) Read the problem again:

it is known that the process starts in state $X_0=1$

This means $$P(X_0=1) =1.$$

$\endgroup$
  • $\begingroup$ Why is the answer getting down vote? It seems correct to me. $\endgroup$ – ABC Oct 1 '15 at 20:37
  • $\begingroup$ @ABC It was incomplete before, I only answered the second question. $\endgroup$ – Antitheos Oct 1 '15 at 20:38
  • 1
    $\begingroup$ But I think we need to welcome an answer if we get a correct answer of one of the several questions. Something is better than nothing. $\endgroup$ – ABC Oct 1 '15 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.