# Solving a Markov Chain

Let the distribution on variables $$(X_t)$$ for $$t \in N$$ satisfy a Markov chain. Each variable can take the values $$\{1, 2\}$$. We are given the pmfs $$p(X_1=i) = 0.5$$ for $$i=1,2$$ and $$p(X_{t+1} = j\mid X_t = i) = p_{i,j}$$ where $$p_{i,j}$$ is the $$(i, j)$$-th element of the matrix $$P=\begin{pmatrix} 0.3 & 0.7\\ 0.6 & 0.4 \end{pmatrix}$$ Find: $$P(X_3 = 2)$$ and $$p(X_2 = 1\mid X_3 = 2)$$.

I'm stuck with how to start this problem. So any hints would be appreciated.

Hints: By the law of total probability: $$P(X_3=2)=\sum_{i,j\in \{1,2\}}P(X_3=2\mid X_1=i,X_2=j)P(X_1=i,X_2=j)$$ But by the Markov property, $$X_3$$ is independent of $$X_1$$, hence the inner term simplifies to $$P(X_3=2)=\sum_{i,j\in \{1,2\}}P(X_3=2\mid X_2=j)P(X_1=i,X_2=j)$$ Now, $$P(X_3=2\mid X_2=j)$$ is easy to compute (right?) and $$P(X_1=i,X_2=j)=P(X_2=j\mid X_1=i)P(X_1=i)$$ which is again easy to compute (right?) for any $$i,j$$.
$$P(X_2=1\mid X_3=2)=\frac{P(X_3=2\mid X_2=1)P(X_2=1)}{P(X_3=2)}$$ and of course use the first part to avoid repeating calculations.