The Markov property for a discrete state time stochastic process is defined as:

$$\mathbb{P}(X_n=x_n\mid X_{n-1}=x_{n-1}, \dots, X_0=x_0)=\mathbb{P}(X_n=x \mid X_{n-1}=x_{n-1})$$

A corollary is

$$\mathbb{P}(x_3 \mid x_0, x_1)=\mathbb{P}(x_3 \mid x_1)$$

We technically ignore state $2$

Now I am having an issue proving this. First I want to make sure certain steps are correct.

  1. I start with $\mathbb{P}(x_3 \mid x_0, x_1) = \sum_{x_2} \mathbb{P}(x_3, x_2 \mid x_0, x_1) $

so $ \ \mathbb{P}(x_3 \mid x_0, x_1) = \sum_{x_2} \mathbb{P}(x_3 \mid x_0, x_1, x_2) \mathbb{P}(x_2 \mid x_0, x_1)$

Is it true that $ \ \sum_{x_2} \mathbb{P}(x_2 \mid x_0, x_1) = 1 \ \ \ (\Psi)$

  1. If $(\Psi)$ is correct, then

$ \ \mathbb{P}(x_3 \mid x_0, x_1) = \mathbb{P}(x_3 \mid x_0, x_1, x_2) $

Is it now true that:

$$ \ \mathbb{P}(x_3 \mid x_0, x_1, x_2) = \mathbb{P}(x_3 \mid x_1, x_2)$$

If yes, why? It appears the M.P. only applies to the last state, and not a combination of states.


I don't see how you come to the conclusion in 2 from $\Psi$. Note that the event on the left hand side $X_2=x_2$ is not properly defined (where does $x_2$ come from?).

If you take another step and apply the Markov property you get: \begin{align*} P(X_3=x_3|X_0=x_0,X_1=x_1) &= \sum_{x_2}P(X_3=x_3|X_0=x_0,X_1=x_1,X_2=x_2)P(X_2=x_2|X_0=x_0,X_1=x_1) \\ &= \sum_{x_2}P(X_3=x_3|X_1=x_1,X_2=x_2)P(X_2=x_2|X_1=x_1) \\ &= \sum_{x_2}P(X_3=x_3,X_2=x_2|X_1=x_1) \\ &= P(X_3=x_3|X_1=x_1)\ . \end{align*}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.