Assume we have a stochastic process $\{X_n\}_\mathbb{N}$ defined on some underlying probability space that takes values in another measurable space. One of the many definitions that I have seen of the Markov property is as follows:

The process has the Markov property iff for arbitrary $n > s$ and $A$ measurable set

$\mathbb{P}(X_n \in A| \: \sigma(X_1,\dots,X_s)) = \mathbb{P}(X_n \in A| \: \sigma(X_s)) \tag{1}$

Is it possible to define the Markov property as

$$\mathbb{P}(X_n \in A| \: \sigma(X_1,\dots,X_{n-1})) = \mathbb{P}(X_n \in A| \: \sigma(X_{n-1}))$$

and then deduce that $(1)$ holds?

Since $\mathbb{P}(X_n \in A| \: \sigma(X_1,\dots,X_{n-1}))$ = $\mathbb{E}(\mathbb{1}_{X_n \in A}|\: \sigma(X_1,\dots,X_{n-1}))$ I have been trying to use properties of conditional expectation but have not been successful.



Yes, they are equivalent. Let's assume that

$$\mathbb{P}(X_n \in A \mid \sigma(X_1,\ldots,X_{n-1}) = \mathbb{P}(X_n \in A \mid \sigma(X_{n-1})) \tag{1}$$

holds for all measurable sets $A$ and all $n \in \mathbb{N}$. By a standard approximation procedure, this implies $$\mathbb{E}(f(X_n) \mid \sigma(X_1,\ldots,X_{n-1}) ) = \mathbb{E}(f(X_n) \mid \sigma(X_{n-1})) \tag{1'}$$ for any bounded Borel-measurable function $f$.

For fixed $n \in \mathbb{N}$ we prove $$\mathbb{P}(X_n \in A \mid \sigma(X_1,\ldots,X_{n-k})) = \mathbb{P}(X_n \in A \mid \sigma(X_{n-k})), \qquad A \in \mathcal{A}, \tag{2}$$ by induction over $k=1,\ldots,n$.

Base: For $k=1$ this is nothing but $(1)$.

Inductive step: Assume that $(2)$ holds for some $k=1,\ldots,j$; we have to show that $(2)$ holds for $k=j+1$. By the tower property of conditional expectation, we have

$$\mathbb{P}(X_{n} \in A \mid \sigma(X_1,\ldots,X_{n-j-1})) = \mathbb{E} \bigg[ \mathbb{P}(X_k \in A \mid \sigma(X_1,\ldots,X_{n-j})) \mid \sigma(X_1,\ldots,X_{n-j-1}) \bigg].$$

Using our induction hypothesis, we find

$$\mathbb{P}(X_{n} \in A \mid \sigma(X_1,\ldots,X_{n-j-1})) = \mathbb{E} \bigg[ \mathbb{P}(X_n \in A \mid \sigma(X_{n-j})) \mid \sigma(X_1,\ldots,X_{n-j-1}) \bigg].$$

By the factorization lemma, there exists a measurable function $f$ such that

$$\mathbb{P}(X_n \in A \mid \sigma(X_{n-j})) = f(X_{n-j}),$$

and so

$$\mathbb{P}(X_{n} \in A \mid \sigma(X_1,\ldots,X_{n-j-1})) = \mathbb{E}(f(X_{n-j}) \mid \sigma(X_1,\ldots,X_{n-j-1})).$$

It follows from $(1')$ that

$$\mathbb{P}(X_{n} \in A \mid \sigma(X_1,\ldots,X_{n-j-1})) = \mathbb{E}(f(X_{n-j}) \mid \sigma(X_{n-j-1})). \tag{3}$$

If we take on both sides the conditional expectation with respect to $\sigma(X_{n-j-1})$, then we find that

$$\mathbb{P}(X_n \in A \mid \sigma(X_{n-j-1})) = \mathbb{E}(f(X_{n-j}) \mid \sigma(X_{n-j-1})). \tag{4}$$

Combining $(3)$ and $(4)$ we get

\begin{align*} \mathbb{P}(X_{n} \in A \mid \sigma(X_1,\ldots,X_{n-j-1})) &\stackrel{(3)}{=} \mathbb{E}(f(X_{n-j}) \mid \sigma(X_{n-j-1})) \\ &\stackrel{(4)}{=} \mathbb{P}(X_n \in A \mid \sigma(X_{n-j-1})), \end{align*}

i.e. $(2)$ holds for $k=j+1$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.