# Markov Chain from Martingale

The two parts to this problem show how processes can be characterized using martingales. In each part, let $$(\Omega,\mathcal{F},\mathbb{P})$$ be a probability space and let $$\{\mathcal{F}_n\}$$ be a filtration. Let $$E$$ be a finite or countable collection of points and $$P=(p_{ij})=(p(i,j))$$ be an $$E\times E$$ stochastic matrix. For each bounded function $$f$$ on $$E$$ define $$Af(i)=\sum_j(f(j)-f(i))p(i,j)=\sum_jf(j)p(i,j)-f(i)$$

Let $$f$$ be a bounded function on $$E$$. Define the process $$M$$ such that $$M_0=0$$ and for $$n\in \mathbb{N}$$,

$$M_n=f(Z_n)-f(Z_0)-\sum_{k=0}^{n-1}Af(Z_k)$$ 1) Let $$Z=\{Z_n:n=0,1,2,...\}$$ be a Markov chain having initial distributions $$\mu$$ and transition probabilities $$P$$; i.e., $$\mathbb{P}(Z_{n+1}=j|\mathcal{F}_n)=p(Z_n,j)$$.

The first part was to show that $$M$$ is a Martingale (what I typed above), which I already did. I am struggling with the second part, which says:

Now suppose that $$Z=\{Z_0,Z_1,Z_2,...\}$$ is a sequence of random variables such that $$M$$ is an $$\{\mathcal{F}_n\}$$-martingale for every bounded function $$f$$ on $$E$$, where $$M_n$$ is defined above and $$\mathbb{P}(Z_0=i)=\mu(\{i\})$$. Show that $$Z$$ is a Markov chain having transition probability matrix $$P$$.

I know I need to show that

$$\mathbb{P}(Z_0=i_0,Z_1=i_1,...,Z_n=i_n)=\mu_{io}p_{i_0i_1}...p_{i_{n-1}}p_{i_n}$$

that is, that every state is independent up to n.

My first idea is that since $$M$$ is a Martingale and $$\mathbb{E}[M_n|\mathcal{F}_{n-1}]=M_{n-1}$$, we can transfer that to $$Z_n$$, namely, $$\mathbb{E}[Z_n|\mathcal{F}_{n-1}]=Z_{n-1}$$ but after that I don't really know how to proceed.

I would appreciate any help!

Thanks.

Given any bounded function $$f$$, we have

\begin{align*} \mathbb E[f(Z_n)|\mathcal F_{n-1}] &= \mathbb E[M_n | \mathcal F_{n-1}] + f(Z_0) + \sum_{k=0}^{n-1}Af(Z_k) \\ &= M_{n-1} + f(Z_0) + \sum_{k=0}^{n-1}Af(Z_k) \\ &= f(Z_{n-1}) + Af(Z_{n-1}) \\ &= \sum_j f(j)p(Z_{n-1},j). \end{align*}

Apply this for the function $$f=\mathbf 1_{\{z\}}$$ to obtain

$$\mathbb P(Z_n = z|\mathcal F_{n-1}) = \mathbb E[f(Z_n)|\mathcal F_{n-1}] = \sum_jf(j)p(Z_{n-1},j) =p(Z_{n-1},z).$$

This shows $$\{Z_n\}$$ is a Markov chain with transition probabilities $$p$$.