Let the distribution on variables $(X_t)$ for $t \in N$ satisfy a Markov chain. Each variable can take the values $\{1, 2\}$. We are given the pmfs $$p(X_1=i) = 0.5$$ for $i=1,2$ and $$p(X_{t+1} = j\mid X_t = i) = p_{i,j}$$ where $p_{i,j}$ is the $(i, j)$-th element of the matrix $$P=\begin{pmatrix} 0.3 & 0.7\\ 0.6 & 0.4 \end{pmatrix}$$ Find: $P(X_3 = 2)$ and $p(X_2 = 1\mid X_3 = 2)$.

I'm stuck with how to start this problem. So any hints would be appreciated.


Hints: By the law of total probability: $$P(X_3=2)=\sum_{i,j\in \{1,2\}}P(X_3=2\mid X_1=i,X_2=j)P(X_1=i,X_2=j)$$ But by the Markov property, $X_3$ is independent of $X_1$, hence the inner term simplifies to $$P(X_3=2)=\sum_{i,j\in \{1,2\}}P(X_3=2\mid X_2=j)P(X_1=i,X_2=j)$$ Now, $P(X_3=2\mid X_2=j)$ is easy to compute (right?) and $$P(X_1=i,X_2=j)=P(X_2=j\mid X_1=i)P(X_1=i)$$ which is again easy to compute (right?) for any $i,j$.

For the second part, just use Bayes rule:

$$P(X_2=1\mid X_3=2)=\frac{P(X_3=2\mid X_2=1)P(X_2=1)}{P(X_3=2)}$$ and of course use the first part to avoid repeating calculations.

  • 1
    $\begingroup$ @james15c Sorry, I do not know the answer, I have not done the calculations. $\endgroup$ – Jimmy R. Nov 7 '18 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.