If $Z$ is i.i.d. and independent of $X_0$, then $X_n:=f(X_{n-1},Z_n)$ is Markov

Let

• $$(E_i,\mathcal E_i)$$ be a measurable space;
• $$f:E_1\times E_2\to E_1\to E_1$$ be $$(\mathcal E_1\otimes\mathcal E_2,\mathcal E_1)$$-measurable;
• $$(\Omega,\mathcal A,\operatorname P)$$ be a probability space;
• $$X_0$$ be an $$E_1$$-valued random variable on $$(\Omega,\mathcal A,\operatorname P)$$;
• $$(Z_n)_{n\in\mathbb N}$$ be an $$E_2$$-valued i.i.d. process on $$(\Omega,\mathcal A,\operatorname P)$$ independent of $$X_0$$ and $$\mu$$ denote the distribution of $$Z_1$$.

How do we show that $$X_n:=f(X_{n-1},Z_n)\;\;\;\text{for }n\in\mathbb N$$ is a time-homogeneous Markov chain with transition kernel $$(\kappa g)(x):=\operatorname E\left[(g\circ f)(x,Z_1)\right]=\int(g\circ f)(x,z)\:\mu({\rm d}z)\;\;\;\text{for }x\in E_1$$ for bounded $$\mathcal E_1$$-measurable $$g:E_1\to\mathbb R$$?

Remember the following result:

Lemma: Let $$Y_i$$ be an $$(E_i,\mathcal E_i)$$-valued random variable on $$(\Omega,\mathcal A,\operatorname P)$$, $$h:E_1\times E_2\to\mathbb R$$ is bounded and $$(\mathcal E_1\otimes\mathcal E_2)$$-measurable and $$\mathcal F\subseteq\mathcal A$$ be a $$\sigma$$-algebra. If $$\mathcal F$$ is independent of $$Y_2$$ and $$Y_1$$ is $$\mathcal F$$-measurable, then $$\operatorname E\left[h(Y_1,Y_2)\mid\mathcal F\right]=\left.\operatorname E\left[h(y_1,Y_2)\right]\right|_{y_1\:=\:Y_1}\tag1.$$

Using this lemma, it is easy to see that $$\operatorname E\left[g(X_{n+1})\mid\mathcal F^X_n\right]=(\kappa g)(X_n)\tag2$$ holds true for $$n=0$$.

But how do we need to proceed for $$n\ge1$$? Are we somehow able to show that $$\mathcal F^X_n$$ is independent of $$Z$$?

• Notice that $\sigma(X_0, \dots, X_{n-1}) \subseteq \sigma(X_0, Z_1, \dots, Z_{n-1})$ which by assumption is independent of $\sigma(Z_n, Z_{n+1}, \dots)$, and in particular is independent of $Z_n$. Apr 10, 2022 at 18:07
• @NateEldredge Thank you. It's trivial. That was the easy argument I was looking for. Apr 10, 2022 at 18:31

For convenicen Let $$Z'_n=Z_{n-1}$$ for $$n\geq1$$ so that $$X_n(\omega)=f(X_{n-1}(\omega),Z'_{n-1}(\omega))$$

It suffices to show that for any measurable $$A\subset E_1$$, $$E[\mathbb{1}_A(X_{n+1})|\sigma(X_0,\ldots,X_n)=G_n(X_n)$$ for each $$n\geq1$$.

It is easy to see that there are measurable functions $$F_n:E_1\times E^n_2\rightarrow E_1$$ such that $$X_n=F_n(X_0,Z'_0,\ldots,Z'_{n-1})$$ Fix $$n\in\mathbb{N}$$. Any bounded measurable function $$H(\omega):=h(X_0(\omega),\ldots,X_n(\omega))$$ depends on $$X_0,\ldots,X_n$$ only through $$(X_0,Z'_1,\ldots,Z'_{n-1})$$, that is $$H=\Phi(X_0,Z'_0,\ldots,Z'_{n-1})$$. As $$X_0$$ and $$Z'$$ are independent

\begin{align} E[\mathbb{1}_A((X_{n+1})H]&=E[\mathbb{1}_A(F_n(X_0,Z'_0,\ldots,Z'_n))\Phi(X_0,Z'_0,\ldots,Z'_{n-1})]\\ &=\int_{E_1\times E^{n+1}_2 }\mathbb{1}_A(F_n(x_0,z_0,\ldots,z_{n-1},z_n))\Phi(x_0,z_0,\ldots,z_{n-1})\mu_{X_0}(dx_0)\mu^{\otimes n+1}_{Z'_0}(dz_0,\ldots,dz_n)\\ &=\int_{E_0\times E^n_2}\Phi(x_0,z_0,\ldots,z_{n-1})\cdot\\ &\qquad\Big(\int_{E_2}\mathbb{1}_A(F_n(x_0,z_0,\ldots,z_{n-1},z_n))\,\mu_{Z_0}(dz_n)\Big)\,\mu_{X_0}(dx_0)\times\mu^{\otimes n}_{Z_0}(dz_0,\ldots,dz_n)\\ &=\int\Phi(x_0,z_0,\ldots,z_{n-1})\Big(\int\mathbb{1}_A(f(x_n,z_n)\,\mu_{Z_0}(dz_n)\Big)\mu_{X_0}(dx_0)\times\mu^{\otimes n}_{Z_0}(dz_0,\ldots,dz_n)\\ &=E[HG_n(X_n)] \end{align} where $$G(x_n)=\int\mathbb{1}_A(f(x_n,z)\mu_{Z_0}(dz)$$. That is, $$E[\mathbb{1}_A(X_{n+1})|\sigma(X_0,X_1,\ldots, X_n)]=G(X_n)$$