# Why is the Markov property implied by the existence of a transition matrix?

If $\left(X_n\right)_{n\in\mathbb{N}_0}$ is an $E$-valued stochastic process with distributions $\left(P_x\space:\space x\in E\right)$ satisfying $$\mathrm{P}_x\left(X_0=x\right)=1$$ and stochastic kernel $\kappa\left(x,B\right)=\mathrm{P}_x\left(X\in B\right)$, then the Markov property (namely "for every $A\in\mathcal{B}\left(E\right)$, every $x\in E$ and all $i,j\in\mathbb{N}_0$ we have $\mathrm{P}_x\left(\left.X_{i+j}\in A\space\right|\mathcal{F}_i\right)=\kappa_j\left(X_i,A\right)$") is implied if for every $A\in\mathcal{B}\left(E\right)$, every $x\in E$ and every $i\in\mathbb{N}_0$ we have $$\mathrm{P}_x\left(\left.X_{i+1}\in A\space\right|\mathcal{F}_i\right)=\kappa_1\left(X_i, A\right)$$ where $$\kappa_j\left(x, A\right):=\mathrm{P}_x\left(X_j\in A\right)$$ The following fact, easily derived from the given data, can be used without proof: $$\kappa_{n+1}=\kappa_n\cdot\kappa_1:=\intop_E\kappa_n\left(\cdot, \mathrm{d}x\right)\kappa_1\left(x,\cdot\right)$$ with $\kappa_0\left(x,\cdot\right):=\delta_x\left(\cdot\right)$, the Dirac measure.

This is Theorem 17.11 from Klenke's book (p. 350, Chapter 17: "Markov Chains")

Hint

If I understand correctly, all that needs be shown is that for all $m\in\mathbb{N}_0$, $$\mathrm{P}_x\left(\left.X_{i+m}\in A\space\right|\mathcal{F}_i\right)=\kappa_m\left(X_i, A\right)$$

-

Induction over $j\geqslant1$. The case $j=1$ is the hypothesis. Assume that the property holds for some $j\geqslant1$. Then $\mathrm P(X_{i+j+1}\in A\mid \mathcal F_{i+1})=\kappa_j(X_{i+1},A)$ and $\mathcal F_i\subseteq \mathcal F_{i+1}$, hence, by the tower property, $$\mathrm P(X_{i+j+1}\in A\mid \mathcal F_{i})=\mathrm E(\kappa_j(X_{i+1},A)\mid \mathcal F_i).$$ For every suitable function $u$, $\mathrm E(u(X_{i+1})\mid \mathcal F_i)=v(X_i)$, where $$v(\,\cdot\,)=\int u(y)\kappa_1(\,\cdot\,,dy).$$ Applying this to $u=\kappa_j(\cdot,A)$, one gets $$v(\,\cdot\,)=\int \kappa_j(y,A)\kappa_1(\,\cdot\,,dy)=(\kappa_1\cdot \kappa_j)(\,\cdot\,,A).$$ Note that $\kappa_{j+1}=\kappa_j\cdot \kappa_1$ is also $\kappa_{j+1}=\kappa_1\cdot \kappa_j$, hence the RHS above is $\kappa_{j+1}(\,\cdot\,,A)$, which proves as desired that $$\mathrm P(X_{i+j+1}\in A\mid \mathcal F_{i})=\kappa_{j+1}(X_i,A).$$
Thank you very much! It's taken me a while to understand why $\mathrm E(u(X_{i+1})\mid \mathcal F_i)=v(X_i)$ and why $\kappa_j\cdot\kappa_1=\kappa_1\cdot\kappa_j$, but i get it now. – Evan Aad Sep 2 '12 at 17:45