# Simple Markov property

I want to prove the simple Markov property but I come to a point where I do not see how to conclude.

I want to prove $\mathbb{E}_\nu[Z\circ\Theta_t\mid \mathcal{Y}^0_t]=\mathbb{E}_{Y_t}[Z]:=\mathbb{E}_x[Z]\mid_{x=Y_t}\ \mathbb{P}_\nu -a.s.$

for all $Z\geq0$ $\mathcal{Y}_\infty^0=\sigma(Y_s,s\geq0)$-measurable on the path space $S^{[0,\infty)}$ where $S$ is a state space, $\mathcal{Y_t}^0=\sigma(Y_s,s\leq t)$, $Y$ the coordinate process ($Y_t:S^{[0,\infty)}\rightarrow S, y\mapsto y(t)$), $\nu$ the initial distribution and $(K_t)_{t\geq0}$ the transition semi group of the process.

By the following property of the conditional expectation:

$X\in L^1(\mathcal{F}),\ X'\in L^1(\mathcal{G}),\ \mathcal{G}\subset\mathcal{F}:\forall \Lambda\in \mathcal{G}\ \mathbb{E}[X,\Lambda]=\mathbb{E}[X',\Lambda]\Rightarrow X'=\mathbb{E}[X\mid\mathcal{G}]$.

So using this we only have to check that: $\mathbb{E}_\nu[\mathbb{E}_{Y_t}[Z]U]=\mathbb{E}_\nu[(Z\circ\Theta_t)U]\ \forall U \in \mathcal{Y}_t^0, U\geq0$.

By a monoton class argument it is enough to prove it only for $U=\prod_{i=0}^n{f_i(Y_{t_i})}$ with $t_n\leq t$ and $Z=\prod_{k=0}^m{g_k(Y_{s_k})}$

So now we get following

$\mathbb{E}_\nu[(\prod_{k=0}^m{g_k(Y_{s_k})}\circ\Theta_t)\prod_{i=0}^n{f_i(Y_{t_i})}]=\mathbb{E}_\nu[\prod_{k=0}^m{g_k(Y_{s_k+t})}\prod_{i=0}^n{f_i(Y_{t_i})}]$

And ordering increasingly the terms and since $Y$ is Markov we get

$=\int_{S^{m+n+2}}{\nu(dy_{t_0})f_0(y_{t_0})}\cdots f_n(y_{t_n})K_{t_n-t_{n-1}}(y_{t_{n-1}},dy_{t_n})g_0(y_{s_0+t})K_{s_0+t-t_n}(y_{t_n},dy_{s_0+t})\cdots g_m(y_{s_m})K_{s_{m-1}-s_m}(y_{s_{m-1}+t},dy_{s_m+t})$

Now, at this point I found a proof for the discrete case and I applied the same idea, which consists to set some how $z_{s_k}=y_{s_k+t}$ to get following

$=\int_{S^{n+1}}{\nu(dy_{t_0})f_0(y_{t_0})}\cdots f_n(y_{t_n})K_{t_n-t_{n-1}}(y_{t_{n-1}},dy_{t_n})\int_{S^{m+1}}g_0(z_{s_0})K_{s_0+t-t_n}(y_{t_n},dy_{s_0+t})\cdots g_m(z_{s_m})K_{s_{m-1}-s_m}(z_{s_{m-1}},dz_{s_m})$

$=\int_{S^{n+1}}{\nu(dy_{t_0})f_0(y_{t_0})}\cdots f_n(y_{t_n})K_{t_n-t_{n-1}}(y_{t_{n-1}},dy_{t_n})\mathbb{E}_{y_{t_n}}[\prod_{k=0}^m{g_k(Y_{s_k})}]=\mathbb{E}[\prod_{i=0}^n{f_i(Y_{t_i})}\mathbb{E}_{Y_{t_n}}[\prod_{k=0}^m{g_k(Y_{s_k})}]]$

And so the proof should be concluded. My problems are the change of variables to cancel $t$ in the integrals and the $\mathbb{E}_{y_{t_n}}$ which I have trouble to work with. Moreover the passage to $\mathbb{E}_{Y_{t_n}}$ I'm thinking if it is really that trivial.

Thanks for any help :)

Let $v$ be a probability measure and $(P_{s,t})_{0\le s \le t}$ be a transition function on a measurable space $(E,\mathcal{E})$ in which $E$ is polish and $\mathcal{E}$ is its Borel $\sigma$ field. Let $\Omega=E^{[0,+\infty)}$, $\mathcal{F}=\mathcal{E}^{[0,+\infty)}$. Let $\theta_t$ be a shift operator on $\Omega$ and $Z$ be any non-negative $\mathcal{F}/\mathcal{B}(\mathbb{R})$ measurable random variable on $(\Omega,\mathcal{F},P)$.

Let's first verify:

(i) $E_x(Z)$ as a function of $x\in E$ is $\mathcal{E}/\mathcal{B}(\overline{\mathbb{R}})$ measurable and $E_v(Z)=\int_E E_x(Z)v(dx)$.

Then we will show:

(ii) if $P_{s,t}$ is homogeneous, then $\forall t>0$, $E_v(Z(\theta_t)|\mathcal{F}_t)=E_{X_t}(Z)$ a.s.-$P_v$.

## Proof of (i):

Let $\mathcal{D}$ to be the collection of all $\mathcal{F}$ measurable sets $G$ such that the indicator function $I_G$ satisfies (i), that is $\mathcal{D}=\{G\in\mathcal{F}|E_x(I_G) \mbox{ as a function of } x \mbox{ is }\mathcal{E}/\mathcal{B}(\mathbb{R})\mbox{ measurable and }E_v(I_G)=\int_E E_x(I_G)v(dx)\}$.

Define $$\mathcal{I}=\{U=\{X_0\in A_0,X_{t_1}\in A_1,\dots,X_{t_n}\in A_n\}| n\ge0,0<t_1<\cdots<t_n, A_0,A_1,\dots,A_n\in\mathcal{E}\}.$$ Clearly, $\mathcal{I}$ is a $\pi$ system on $\Omega$ and $\sigma(\mathcal{I})=\sigma(X_t,t\ge0)=\mathcal{E}^{[0,+\infty)}=\mathcal{F}$.

Step 1: Show $\mathcal{D}$ is a d system on $\Omega$.\ First of all, $\Omega\in\mathcal{F}$ and $I_\Omega\equiv 1$ so $E_x(I_\Omega)=1$ ($\forall x\in E$) thus it is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$. And $\int_E E_x(I_\Omega)v(dx)=\int_E v(dx)=v(E)=1=E_v(I_\Omega)$ so $\Omega\in\mathcal{D}$ by the definition of $\mathcal{D}$.

Next, suppose $G,H\in\mathcal{D}$ such that $H\subseteq G$ ($G\setminus H\in\mathcal{F}$ because $G,H\in\mathcal{F}$). Notice that $I_{G\setminus H}=I_G-I_H$ so $E_x(I_{G\setminus H})=E_x(I_G)-E_x(I_H)$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ (because $E_x(I_G),E_x(I_H)$ are both $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$.) And $E_v(I_{G\setminus H})=E_v(I_G)-E_v(I_H)=\int_E E_x(I_G)v(dx)-\int_E E_x(I_H)v(dx)=\int_E E_x(I_G-I_H)v(dx)=\int_E E_x(I_{G\setminus H})v(dx)$. This shows $G\setminus H\in\mathcal{D}$.

Lastly, suppose $\{G_n\}_{n=1}^\infty$ is a non-decreasing sequence in $\mathcal{D}$ (so $\cup_{n=1}^\infty G_n \in\mathcal{F}$ because each $G_n\in\mathcal{F}$). Note that $\underset{n\to\infty}{\lim}I_{G_n}(\omega)=I_{\cup_{n=1}^\infty}G_n(\omega)$). Since $G_n$ is non-decreasing in $n$,$I_{G_n}(\omega)$ is non-negative and non-decreasing in $n$ so by monotone convergence theorem, $E_x(I_{\cup_{n=1}^\infty G_n})=E_x(\underset{n\to\infty}{\lim}I_{G_n})=\underset{n\to\infty}{\lim} E_x(I_{G_n})$, which is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ because $E_x(I_{G_n})$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ for each $n$. In addition, $E_v(I_{\cup_{n=1}^\infty G_n})=E_v(\underset{n\to\infty}{\lim} I_{G_n})\overset{(*)}{=}\underset{n\to\infty}{\lim}E_v(I_{G_n})=\underset{n\to\infty}{\lim}\int_E E_x(I_{G_n})v(dx)\overset{(**)}{=}\int_E \underset{n\to\infty}{\lim} E_x(I_{G_n})v(dx)\overset{(*)}{=}\int_E E_x(\underset{n\to\infty}{\lim}I_{G_n})v(dx)=\int_E E_x(I_{\cup_{n=1}^\infty G_n})v(dx)$ where (*) and (**) used monotone convergence theorem on $I_{G_n}$ and $E_x(I_{G_n})$ respectively ($E_x(I_{G_n})$ as functions of $x$ are non-decreasing in $n$ and non-negative). Hence, $\cup_{n=1}^\infty G_n \in \mathcal{D}$. Thus $\mathcal{D}$ is a d system.

Step 2: show $\mathcal{I}\subseteq\mathcal{D}$. Let $U\in\mathcal{I}$ so $U=\{X_0\in A_0,X_{t_1}\in A_1,\dots, X_{t_n}\in A_n\}$ for some $n\ge0$, $0<t_1<t_2<\cdots<t_n$ and $A_0,A_1,A_2,\dots, A_n\in\mathcal{E}$ (clearly $U\in\mathcal{F}$).

For $x\in E$, $X_t$ as a stochastic process defined on $(\Omega,\mathcal{F},P_x)$ is markov with initial distribution $\delta_x$ and transition function $P_{s,t}$. $X_t$ as a process defined on $(\Omega,\mathcal{F},P_v)$ is a markov process with transition function $P_{s,t}$ and initial distribution $v$. Now, $E_x(I_U)=E_x(I_{A_0}(X_0)I_{A_1}(X_{t_1})I_{A_2}(X_{t_2})\cdots I_{A_n}(X_{t_n}))=E_x(I_{A_0}(X_0)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(X_0))$ $$=\int_E I_{A_0}(y)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(y)\delta_x(dy)=I_{A_0}(x)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(x)\quad\quad(1)$$ (when taking $E_x(\bullet)$, $X_t$ is assumed to be defined on $(\Omega,\mathcal{F},P_x)$ so $\delta_x$ is the initial distribution)\ The right hand side of (1) as a function of $x$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable ( because $I_{A_0},\dots I_{A_n}$ are $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable and we know transition probabilities turn $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable functions into $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable functions) so $E_x(I_U)$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$.

Similarly, let's now calculate $E_v(I_U)$. When taking the expectation $E_v(\bullet)$, markov process $X_t$ is assumed to be defined on $(\Omega,\mathcal{F},P_v)$ so $v$ is the initial distribution. So $E_v(I_U)=E_v(I_{A_0}(X_0)I_{A_1}(X_{t_1})I_{A_2}(X_{t_2})\cdots I_{A_n}(X_{t_n}))=\int_E I_{A_0}(x)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(x)v(dx)$ $\overset{\mbox{by (1)}}{=}\int_E E_x(I_U)v(dx)$. Thus $U\in\mathcal{D}$, this finishes the proof of step 2.

Step 3: show for any $G\in\mathcal{F}$, $E_x(I_G)$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ and $E_v(I_G)=\int_E E_x(I_G)v(dx)$.\ Since $\sigma(I)=\mathcal{F}$, $\mathcal{I}\subseteq\mathcal{D}$ so $\mathcal{D}\supseteq\mathcal{F}$ (monotone class theorem). By definition of $\mathcal{D}$, step 3 is proved.

Step 4: show that if $Z$ is a $\mathcal{F}/\mathcal{B}(\mathbb{R})$ measurable non-negative simple function from $\Omega$ to $\mathbb{R}$, then $E_x(Z)$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ and $E_v(Z)=\int_E E_x(Z)v(dx)$.

Since $Z$ is non-negative simple, we can write $Z=\sum_{i=1}^k a_i I_{G_i}$ for some non-negative real numbers $a_1,a_2,\dots,a_k$ and $G_1,G_2,\dots,G_n\in\mathcal{F}$. $E_x(Z)=\sum_{i=1}^k a_i E_x(I_{G_i})$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ because by step 3, $E_x(I_{G_i})$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ for each $i$. In addition using step 3 on $E_v(I_{G_i})$, we have $E_v(Z)=\sum_{i=1}^k a_i E_v(I_{G_i})=\sum_{i=1}^k a_i \int_E E_x(I_{G_i})v(dx)=\int_E E_x(\sum_{i=1}^k a_i I_{G_i})v(dx)=\int_E E_x(Z)v(dx)$. This finishes the proof of step 4.

Now, we are finally ready to prove the result. suppose $Z$ is non-negative and $\mathcal{F}/\mathcal{B}(\mathbb{R})$ measurable. There is a sequence $(Z_n)_{n=1}^\infty$ of non-negative simple $\mathcal{F}/\mathcal{B}(\mathbb{R})$ measurable functions increasing to $Z$. By monotone convergence theorem, $E_x(Z)=E_x(\underset{n\to\infty}{\lim}Z_n)=\underset{n\to\infty}{\lim}E_x(Z_n)$ which is $\mathcal{E}/\mathcal{B}(\overline{\mathbb{R}})$ measurable in $x$ (since by step 3, $E_x(Z_n)$ is $\mathcal{E}/\mathcal{B}(\mathbb{R})$ measurable in $x$ for each $n$ so its limit is $\mathcal{E}/\mathcal{B}(\overline{\mathbb{R}})$ measurable. The limit could be $+\infty$ so we allow $E_x(Z)=+\infty$).

In addition, we have $E_v(Z)=E_v(\underset{n\to\infty}{\lim} Z_n)\overset{(2)}{=}\underset{n\to\infty}{\lim} E_v(Z_n)\overset{\mbox{by step 4}}{=}\underset{n\to\infty}{\lim}\int_E E_x(Z_n)v(dx)\overset{(3)}{=}\int_E \underset{n\to\infty}{\lim} E_x(Z_n)v(dx)\overset{(2)}{=}\int_E E_x(\underset{n\to\infty}{\lim}Z_n)v(dx)=\int_E E_x(Z)v(dx)$ where (2) used monotone convergence theorem on $Z_n$ and (3) used monotone convergence theorem on $E_x(Z_n)$ (note that $E_x(Z_n)$ as functions of $x$ are also non-negative and non-decreasing in $n$ because $Z_n$'s are).

## Proof of (ii):

Let $\mathcal{I}$ be the same as defined in (i) so it's a $\pi$ system on $\Omega$ and $\sigma(\mathcal{I})=\mathcal{F}$. Define $\mathcal{D}'=\{G\in\mathcal{F}| E_v(I_G(\theta_t)|\mathcal{F}_t)=E_{X_t}(I_G)\quad \mbox{a.s.-}P_v\}$. We can show $\mathcal{D}'$ is a d system on $\Omega$ similarly as in (i).

Now, we prove $\mathcal{I}\subseteq\mathcal{D}'$. Let $U\in\mathcal{I}$ so $U=\{X_0\in A_0,X_{t_1}\in A_1,\dots, X_{t_n}\in A_n\}$ for some $n\ge0$, $0<t_1<t_2<\cdots<t_n$ and $A_0,A_1,A_2,\dots, A_n\in\mathcal{E}$ (clearly $U\in\mathcal{F}$). Obviously, $$I_U(\omega)=I_{\{X_0\in A_0\}}(\omega)I_{\{X_{t_1}\in A_1\}}(\omega)\cdots I_{\{X_{t_n}(\omega)\in A_n\}}(\omega)=I_{A_0}(X_0(\omega))I_{A_1}(X_{t_1}(\omega))\cdots I_{A_n}(X_{t_n}(\omega))$$ and by (ii) we have $$I_U(\theta_t(\omega))=I_{\{X_t\in A_0\}}(\omega)I_{\{X_{t_1+t}\in A_1\}}(\omega)\cdots I_{\{X_{t_n+t}(\omega)\in A_n\}}(\omega)=I_{A_0}(X_t(\omega))I_{A_1}(X_{t_1+t}(\omega))\cdots I_{A_n}(X_{t_n+t}(\omega))$$ On $(\Omega,\mathcal{F},P_v)$, $X_t$ is markov with initial distribution $v$ and transition function $P_{s,t}$, thus $$E_v(I_U(\theta_t)|\mathcal{F}_t)=E_v(I_{A_0}(X_t)\prod_{i=1}^n I_{A_i}(X_{t_i+t})|\mathcal{F}_t)=I_{A_0}(X_t)E_v(\prod_{i=1}^n I_{A_i}(X_{t_i+t})|\mathcal{F}_t)$$ $$=I_{A_0}(X_t)E_v(E_v(\prod_{i=1}^n I_{A_i}(X_{t_i+t})|\mathcal{F}_{t_{n-1}+t})|\mathcal{F}_t) \quad (\mbox{because }\mathcal{F}_t\subseteq\mathcal{F}_{t_{n-1}+t})$$ $$=I_{A_0}(X_t)E_v(\prod_{i=1}^{n-1} I_{A_i}(X_{t_i+t})E_v( I_{A_n}(X_{t_n+t})|\mathcal{F}_{t_{n-1}+t})|\mathcal{F}_t)$$ $$=I_{A_0}(X_t)E_v(\prod_{i=1}^{n-1} I_{A_i}(X_{t_i+t})P_{t_{n-1}+t,t_n+t}I_{A_n}(X_{t_{n-1}+t})|\mathcal{F}_t)$$ $$\cdots\quad\cdots\quad\cdots (\mbox{conditioning on }\mathcal{F}_{t_{n-2}+t},\mathcal{F}_{t_{n-3}+t},\cdots,\mathcal{F}_{t_1+t}\mbox{ and repeat the previous steps})$$ $$=I_{A_0}(X_t)E_v(I_{A_1}(X_{t_1+t})P_{t_1+t,t_2+t}I_{A_2}\cdots P_{t_{n-1}+t,t_n+t}I_{A_n}(X_{t_1+t})|\mathcal{F}_t)$$ $$=I_{A_0}(X_t)P_{t,t_1+t}I_{A_1}P_{t_1+t,t_2+t}I_{A_2}\cdots P_{t_{n-1}+t,t_n+t}I_{A_n}(X_t) \quad \mbox{a.s.-}P_v \quad (4)$$ where the last step in (4) used markov property on $f(X_{t_1+t})$ for $f(x)=I_{A_1}(x)P_{t_1+t,t_2+t}I_{A_2}\cdots P_{t_{n-1}+t,t_n+t}I_{A_n}(x)$.

On the other hand, for $x\in E$, $X_t$ as a stochastic process defined on $(\Omega,\mathcal{F},P_x)$ is markov with initial distribution $\delta_x$ and transition function $P_{s,t}$, so, we have $$E_x(I_U)=E_x(I_{A_0}(X_0)I_{A_1}(X_{t_1})\cdots I_{A_n}(X_{t_n}))$$ $$=\int_E I_{A_0}(y)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(y)\delta_x(dy)= I_{A_0}(x)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(x)$$ So in the above equation, plugging $X_t$ into $x$, we have $$E_{X_t}(I_U)= I_{A_0}(X_t)P_{0,t_1}I_{A_1}P_{t_1,t_2}I_{A_2}\cdots P_{t_{n-1},t_n}I_{A_n}(X_t)\quad\quad(5)$$ By (4)(5) and $P_{s,t}$ is homogeneous, $E_v(I_U(\theta_t)|\mathcal{F}_t)=E_{X_t}(I_U)$ a.s.-$P_v$. Thus $\mathcal{I}\subseteq\mathcal{D}'$.

Similar to (i): we then can show (iii) is true for $Z$ being any indicator function of some $G\in\mathcal{F}$ and for $Z$ being any non-negative, simple $\mathcal{F}/\mathcal{B}(\mathbb{R})$ measurable random variable and finally for $Z$ being any non-negative $\mathcal{F}/\mathcal{B}(\mathbb{R})$ measurable random variable.