The two parts to this problem show how processes can be characterized using martingales. In each part, let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space and let $\{\mathcal{F}_n\}$ be a filtration. Let $E$ be a finite or countable collection of points and $P=(p_{ij})=(p(i,j))$ be an $E\times E$ stochastic matrix. For each bounded function $f$ on $E$ define $$Af(i)=\sum_j(f(j)-f(i))p(i,j)=\sum_jf(j)p(i,j)-f(i)$$

Let $f$ be a bounded function on $E$. Define the process $M$ such that $M_0=0$ and for $n\in \mathbb{N}$,

$$M_n=f(Z_n)-f(Z_0)-\sum_{k=0}^{n-1}Af(Z_k)$$ 1) Let $Z=\{Z_n:n=0,1,2,...\}$ be a Markov chain having initial distributions $\mu$ and transition probabilities $P$; i.e., $\mathbb{P}(Z_{n+1}=j|\mathcal{F}_n)=p(Z_n,j)$.

The first part was to show that $M$ is a Martingale (what I typed above), which I already did. I am struggling with the second part, which says:

Now suppose that $Z=\{Z_0,Z_1,Z_2,...\}$ is a sequence of random variables such that $M$ is an $\{\mathcal{F}_n\}$-martingale for every bounded function $f$ on $E$, where $M_n$ is defined above and $\mathbb{P}(Z_0=i)=\mu(\{i\})$. Show that $Z$ is a Markov chain having transition probability matrix $P$.

I know I need to show that


that is, that every state is independent up to n.

My first idea is that since $M$ is a Martingale and $\mathbb{E}[M_n|\mathcal{F}_{n-1}]=M_{n-1}$, we can transfer that to $Z_n$, namely, $\mathbb{E}[Z_n|\mathcal{F}_{n-1}]=Z_{n-1}$ but after that I don't really know how to proceed.

I would appreciate any help!



1 Answer 1


Given any bounded function $f$, we have

\begin{align*} \mathbb E[f(Z_n)|\mathcal F_{n-1}] &= \mathbb E[M_n | \mathcal F_{n-1}] + f(Z_0) + \sum_{k=0}^{n-1}Af(Z_k) \\ &= M_{n-1} + f(Z_0) + \sum_{k=0}^{n-1}Af(Z_k) \\ &= f(Z_{n-1}) + Af(Z_{n-1}) \\ &= \sum_j f(j)p(Z_{n-1},j). \end{align*}

Apply this for the function $f=\mathbf 1_{\{z\}}$ to obtain

$$ \mathbb P(Z_n = z|\mathcal F_{n-1}) = \mathbb E[f(Z_n)|\mathcal F_{n-1}] = \sum_jf(j)p(Z_{n-1},j) =p(Z_{n-1},z).$$

This shows $\{Z_n\}$ is a Markov chain with transition probabilities $p$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .