Transition Probabilities of $P(X_6=7, X_5=3 \mid X_4=1, X_9=3)$

Question

Given that $X_0=3$ and a discrete time Markov Chain with transition probabilities $p(i,j)$, I need to express $$P(X_6=7, X_5=3 \mid X_4=1, X_9=3)$$ in terms of the transition probabilities $p(i,j)$.

So far I have that \begin{align} P(X_6=7, X_5=3 \mid X_4=1, X_9=3) &= P(X_6=7, X_5=3 \mid X_4=1)\\ &=P(X_6=7 \mid X_5 = 3, X_4=1)\, P(X_5=3 \mid X_4=1)\\ &= P(X_6=7 \mid X_5=3) \, P(X_5=3 \mid X_4=1). \end{align} However I'm not sure where to go from here. Specifically, I'm not sure how to use the fact that $X_0=3$.

Answer 1

When you condition on $\{X_4=1\},$ then the unconditional probability $P(X_0 = 3) = 1$ becomes irrelevant.

If the chain is homogeneous, then $P(X_6=7 \mid X_5=3)$ = $P(X_1=7 \mid X_0=3),$ which can be expressed in terms of $p(i,j).$ And similarly for $P(X_5=3 \mid X_4=1).$