# Show some property of a Markov process

Let

• $$(\Omega,\mathcal A,\operatorname P)$$ be a probability space
• $$I\subseteq\mathbb R$$
• $$(\mathcal F_t)_{t\in I}$$ be a filtration on $$(\Omega,\mathcal A)$$
• $$(E,\mathcal E)$$ be a measurable space
• $$X$$ be a $$(E,\mathcal E)$$-valued $$\mathcal F$$-Markov process with transition semigroup $$\left(\kappa_{s,\:t}:s,t\in I\text{ with }s\le t\right)$$

We can show that$$^1$$ $$\operatorname P\left[X_{t_1}\in B_1,\ldots,X_{t_n}\in B_n\right]\\=\operatorname E\left[1_{\left\{\:X_{t_1}\:\in\:B_1\:\right\}}\operatorname E_{X_{t_1}}\left[1_{\left\{\:X_{t_2}\:\in\:B_2\:\right\}}\operatorname E_{X_{t_2}}\left[\cdots\operatorname E_{X_{t_{n-1}}}\left[1_{\left\{\:X_{t_n}\:\in\:B_n\:\right\}}\right]\right]\right]\right]\tag1$$ almost surely for all $$n\in\mathbb N$$, $$t_1,\ldots,t_n\in I$$ with $$t_1\le\cdots\le t_n$$ and $$B_1,\ldots,B_n\in\mathcal E$$.

By definition of a Markov process, $$\operatorname P\left[X_t\in B\mid\mathcal F_s\right]=\kappa_{s,\:t}(X_s,B)\tag2$$ almost surely for all $$B\in\mathcal E$$ and $$s,t\in I$$ with $$s\le t$$.

How can we conclude from $$(1)$$ and $$(2)$$ that $$\operatorname P\left[X_{t_1}\in B_1,\ldots,X_{t_n}\in B_n\right]\\=\int\operatorname P\left[X_{t_0}\in{\rm d}x_0\right]\int_{B_1}\kappa_{t_0,\:t_1}(x_0,{\rm d}x_1)\cdots\int_{B_n}\kappa_{t_{n-1},\:t_n}(x_{n-1},{\rm d}x_n)\tag3$$ almost surely for all $$n\in\mathbb N$$, $$t_0,\ldots,t_n\in I$$ with $$t_0\le\cdots\le t_n$$ and $$B_1,\ldots,B_n\in\mathcal E$$?

$$^1$$ In $$(1)$$ I'm using the notation $$\operatorname E_{\mathcal G}[Y]:=\operatorname E\left[Y\mid\mathcal G\right]$$.

• So where exactly is your problem? Do you know how to prove $(3)$ for $n=2$? (It seems a bit weird that there is a $t_0$ at the right-hand side of $(3)$ whereas there is no $t_0$ on the left-hand side... but I suppose that this is just a typo.)
– saz
Oct 14 '18 at 18:03
• @saz It actually is not a typo. I've seen this equation with $I=[0,\infty)$ and $t_0=0$ and thought it wouldn't make any difference. Oct 14 '18 at 18:16
• Ah I see, sorry, my mistake (I mistook the first integral for $\int_{B_0} P(X_{t_0} \in dx_0)$ and not $\int P(X_{t_0} \in dx_0)$.)
– saz
Oct 14 '18 at 18:22

I will prove $$(3)$$ only for $$n=2$$; for larger $$n$$ you can proceed by iterating the reasoning which I describe below (or, formally, perform a proof by induction).

Using the tower property for conditional expectation, we find from $$(2)$$ that

\begin{align*} \mathbb{E}(1_B(X_t) \mid X_s) = \mathbb{E} \big[ \mathbb{E}(1_B(X_t) \mid \mathcal{F}_s) \mid X_s \big] &\stackrel{(2)}{=} \mathbb{E} \big[ \kappa_{s,t}(X_s,B) \mid X_s \big] \\ &= \kappa_{s,t}(X_s,B) \end{align*}

for any $$s \leq t$$ and $$B \in \mathcal{E}$$. In your notation, this means that

$$\mathbb{E}_{X_s}(1_B(X_t)) = \kappa_{s,t}(X_s,B). \tag{4}$$

Using a standard monotone class argument, it is not difficult to see that this implies

$$\mathbb{E}_{X_s}(h(X_t)) = \int h(y) \, \kappa_{s,t}(X_s,dy) \tag{5}$$

for any bounded measurable function $$h$$. Now fix $$B_1,B_2 \in \mathcal{E}$$ and $$t_0 \leq t_1 \leq t_2$$. By $$(1)$$, we have

\begin{align*} \mathbb{P}(X_{t_1} \in B_1,X_{t_2} \in B_2) &= \mathbb{P}(X_{t_0} \in E, X_{t_1} \in B_1,X_{t_2} \in B_2) \\ &= \mathbb{E} \big[ 1_{\{X_{t_0} \in E\}} \mathbb{E}_{X_{t_0}} \big[ 1_{\{X_{t_1} \in B_1\}} \mathbb{E}_{X_{t_1}} \big[ 1_{\{X_{t_2} \in B_2\}} \big] \big] \big]. \end{align*}

By $$(4)$$, we get

\begin{align*} \mathbb{P}(X_{t_1} \in B_1,X_{t_2} \in B_2) = \mathbb{E} \big[ 1_{\{X_{t_0} \in E\}} \mathbb{E}_{X_{t_0}} \big[ 1_{\{X_{t_1} \in B_1\}} \kappa_{t_1,t_2}(X_{t_1},B_2) \big] \big] \tag{6} \end{align*}

Applying $$(5)$$ for $$h(y) := 1_{B_1}(y) \kappa_{t_1,t_2}(y,B_2)$$, $$s:=t_0$$ and $$t:=t_1$$ shows

$$\mathbb{E}_{X_{t_0}} \big[ 1_{\{X_{t_1} \in B_1\}} \kappa_{t_1,t_2}(X_{t_1},B_2) \big] = \int 1_{B_1}(x_1) \kappa_{t_1,t_2}(y,B_2) \kappa_{t_0,t_1}(X_{t_0},dx_1).$$

Plugging this into $$(6)$$ we obtain that

\begin{align*} \mathbb{P}(X_{t_1} \in B_1,X_{t_2} \in B_2) =\mathbb{E}\left(1_{\{X_{t_0} \in E\}} \int_{B_1} \kappa_{t_1,t_2}(x_1,B_2) \, \kappa_{t_0,t_1}(X_{t_0},dx_1) \right). \end{align*}

Writing all the expressions on the right-hand side in terms of integrals, we get

$$\mathbb{P}(X_{t_1} \in B_1,X_{t_2} \in B_2) = \int_E \left( \int_{B_1} \left( \int_{B_2} \kappa_{t_1,t_2}(x_1,dx_2) \right) \kappa_{t_0,t_1}(x_0,dx_1) \right) \, \mathbb{P}(X_{t_0} \in dx_0)$$

which is $$(3)$$ (for $$n=2$$).