# Verify Markov property

Given a Markov process $$(X_t)_{t \geq 0}$$ I would like to verify that $$P(X_{t_3} \in A | X_{t_2} \in B \; X_{t_1} \in C)= P(X_{t_3} \in A | X_{t_2} \in B)$$ with $$t_3 > t_2 > t_1$$. I know it should be obvious, but I would like to prove thaat using (if possible) only the definition of a Markov process, that is $$E[\phi(X_{t+h})| \mathcal{F}_t]= E[\phi(X_{t+h})|X_t]$$ or equivalently $$P(X_{t+h}| \mathcal{F}_t)= P(X_{t+h}|X_t)$$. It should be really easy, but I just don't get how to use conditional probability to make it work.

• What does ${F}_t$ mean in your question? – soobster Jul 11 '19 at 20:49
• It is the filtration at a time $t$ – tommy1996q Jul 11 '19 at 20:58

## 1 Answer

Firstly, this is not actually true. For example, define an i.i.d. sequence of random variables $$\{X_i\}$$ defined by,

$$X_i = \begin{cases} 1 &\text{ with probability }1/2\\ -1 &\text{ with probability }1/2 \end{cases}.$$

Let $$S_n = \sum_{i=1}^n X_i$$ (here $$S_0 = 0$$). Then $$\{S_n:n\in\mathbb{N}\}$$ is a Markov chain. Let $$t_1 = 1$$,$$t_2 = 2$$ and $$t_3 = 3$$. Let $$C = \{S_1 = 1\}$$, $$B = \{|S_2|\leq 2\}$$ and $$A = \{S_3 = -3\}$$. If at time 1, $$S_1 = 1$$, then it's impossible for $$S_3 = -3$$, so

$$P(S_{t_3} \in A|S_{t_2}\in B,S_{t_1}\in C) = 0.$$

On the other hand, $$|S_2| \leq 2$$ with probability 1, so

$$P(S_{t_3} \in A|S_{t_2}\in B) = P(S_3= -3) = 1/8 \neq 0 = P(S_{t_3} \in A|S_{t_2}\in B,S_{t_1}\in C).$$

I think what you actually wanted to prove was,

$$P(X_{t_3} \in A|X_{t_2},X_{t_1}) = P(X_{t_3} \in A|X_{t_2}).$$

Notice, in the discrete setting, we can write this as,

$$P(X_{t_3} \in A|X_{t_2}=x_2,X_{t_1}=x_1) = P(X_{t_3} \in A|X_{t_2}=x_2),$$

whenever $$P(X_{t_2}=x_2,X_{t_1}=x_1) > 0$$. But we cannot replace $$(x_2,x_3)$$ with non-atomic sets. That's why the definition of the conditional expectation is a general statement about integrals over different filtrations rather than a straight-forward conditional probability as defined in an undergraduate probability course.

We can do this through repeated iteration of the tower property. Notice that,

$$\sigma(X_{t_2}) \subset \sigma(X_{t_2},X_{t_1}) \subset \mathcal{F}_{t_2}.$$

Then,

\begin{align*} P(X_{t_3}|X_{t_2},X_{t_1}) &= E[P(X_{t_3}|X_{t_2},X_{t_1})|\mathcal{F}_{t_2}] \\ &= E[P(X_{t_3}|\mathcal{F}_{t_2})|X_{t_2},X_{t_1}] \\ &= E[P(X_{t_3}|X_{t_2})|X_{t_2},X_{t_1}] \\ &= P(X_{t_3}|X_{t_2}). \end{align*}

Hope that helps!

*edits: Fixed some typos.

• Great answer! Thanks! – tommy1996q Jul 20 '19 at 8:57
• No problem. I remember specifically working through this exact question while I was preparing for prelims a couple years ago, so it was a pleasure to go over that again. – forgottenarrow Jul 20 '19 at 11:11