# Prove that $W_n := (X_n,Y_n)$ is a Markov chain and determine the transition probabilities.

Let $X_n$ be an irreducible, aperiodic, positive recurrent Markov chain $(\lambda,P)$ on a state space $I$, with stationary distribution $\pi$. Let $Y_n$ be Markov$(\pi,P)$, and independent of $X_n$. Let $T_0 = \{n\geq 1 :X_n = Y_n = i_0\}$ for some fixed state $i_0 \in I$.

Prove that $W_n := (X_n,Y_n)$ is a Markov chain and determine the transition probabilities.

So we need to prove that $\mathbb{P}(W_{n+1} = i_{n+1} |W_0 = i_0,\dots,W_n = i_n) = \mathbb{P}(W_{n+1} = i_{n+1} | W_n = i_n)$. Substitution yields us $\mathbb{P}(W_{n+1} = i_{n+1} |W_0 = i_0,\dots,W_n = i_n) = \mathbb{P}((X_{n+1},Y_{n+1}) = i_{n+1} |(X_{0},Y_{0}) = i_0,\dots,(X_{n},Y_{n}) = i_n)$

But how can I continue?

• Hint: If the sigma-algebras generated by $\{A,B\}$ and $\{C,D\}$ are independent then $$P(A\cap C\mid B,D)=P(A\mid B)P(C\mid D).$$ Can you prove this? Can you apply it to your question? – Did Nov 2 '15 at 10:00