# Martingale that is not a Markov process

1. On the internet, it is suggested that $$X_t=\left(\int_0^t X_s \;ds\right)\;dW_{t}$$ is a martingale, but not a Markov process. I understand that the process $$I_t(C)=\int_0^t C_s \; dW_s$$ is a martingale, but I don't see $X_t$ as an Ito's integral. In order to show it is a martingale but not a Markov process, I want to show that $$E(X_t\mid\mathcal{F}_{t-1})=X_{t-1}\neq E(X_t\mid\sigma(X_{t-1})).$$ I don't know how to take the expecation in this case.

Other counterexamples available online might be wrong. (Or, I could be wrong.)

2. Invalid counterexample: Let $e(t)$ be a random variable which is not independent of $y(0)$ and $$y(t)=y(t-1)+e(t)-E{[}e(t)|F(t-1){]}.$$ Then y(t) is a martingale but it is not a Markov process.

I will show that it is both a martingale and a Markov process. Define $X_{t}$ as $$X_{t}=X_{t-1}+e_{t}-E\left(e_{t}|\mathcal{F}_{t-1}\right)$$

Then, $\{X_{t}\}$ is a martingale because $$E(X_{t}|\mathcal{F}_{t-1}) = E\left(X_{t-1}+e_{t}-E\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\mathcal{F}_{t-1}\right) = E(X_{t-1}|\mathcal{F}_{t-1})+E\left(e_{t}|\mathcal{F}_{t-1}\right)-E\left(e_{t}|\mathcal{F}_{t-1}\right) = E(X_{t-1}|\mathcal{F}_{t-1}) = X_{t-1}$$

However, it is also a Markov process. $$E(X_{t}|\sigma(X_{t-1})) = E\left(X_{t-1}+e_{t}-E\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\sigma(X_{t-1})\right) = E(X_{t-1}|\sigma(X_{t-1}))+E\left(e_{t}|\sigma(X_{t-1})\right)-E\left(E\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\sigma(X_{t-1})\right)$$

Since $\sigma(X_{t-1})\subset\mathcal{F}_{t-1}$, by the towering property, $$E\left(\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\sigma(X_{t-1})\right)=E\left(e_{t}|\sigma(X_{t-1})\right).$$

Thus $$E(X_{t}|\sigma(X_{t-1})) = E(X_{t-1}|\sigma(X_{t-1}))+E\left(e_{t}|\sigma(X_{t-1})\right)-E\left(e_{t}|\sigma(X_{t-1})\right) = E(X_{t-1}|\sigma(X_{t-1})$$

Since $X_{t-1}$ is $\sigma(X_{t-1})$-measurable, $E(X_{t-1}|\sigma(X_{t-1}))=X_{t-1}$. Thus $$E(X_{t}|\sigma(X_{t-1}))=X_{t-1}=E(X_{t}|\mathcal{F}_{t-1})$$ Hence $\{X_{t}\}$ is also a Markov process.

3. Invalid counterexample from the textbook answer key. Alison Etheridge's textbook gives an example: $$S_{n+1}=S_{n}+\xi_{n+1}$$ where $\xi_{n}\in\{-1,1\}$ and $\{\xi_{n}\}_{n\geq0}$ are iid. Then, $$P\left(S_{n+1}=k+1\bigg|S_{n}=k\right)=p \text{and }P\left(S_{n+1}=k-1\bigg|S_{n}=k\right)=1-p.$$ The author claims that $\left\{ S_{n}\right\}$ is a Markov process.

Then in the homework answer key, someone gives $\left\{ Z_{n}\right\}$ as a counterexample of a martingale not being a Markov process: $\displaystyle{Z_{n}=\sum_{i=1}^{n}\xi_{i}}$ where $P(\xi_{i}=1)=\frac{1}{2}=P(\xi_{i}=-1)$.

Obviously, $Z_{n+1}=Z_{n}+\xi_{n+1}.$ So what's the difference between this counterexample and the example? Moreover, given $P(\xi_{i}=1)=\frac{1}{2}=P(\xi_{i}=-1)$, I think is both $\left\{ Z_{n}\right\}$ is both a martingale and a Markov process.

• 1. the definition of $X$ is incorrect - it seems that there should be $dX_t$. Could you give a link for this example?  2. and 3. - I think you are right and your calculations are correct (in 2. I assume that $\mathcal F_t$ is a natural filtration of $y$ or $X$ in your notation). In 3. you have a random walk which is a Markov process, and a martingale if its jump has a zero mean.
– SBF
Feb 26, 2012 at 22:22
• @Ilya, Thanks for the editing. The link is here wilmott.com/messageview.cfm?catid=8&threadid=11322 Feb 26, 2012 at 22:36
• Here is another example I mentioned: mathforum.org/kb/… I need a valid counterexample. Thank you! Feb 26, 2012 at 22:39

There is a pervasive mistake in your post, possibly explaining your trouble, which is to believe that $$(X_t)_{t\geqslant0}$$ being a Markov process means that $$\mathrm E(X_t\mid \mathcal F_{t-1})=\mathrm E(X_t\mid X_{t-1})$$ for every $$t\geqslant1$$, where $$\mathcal F_t=\sigma(X_s;0\leqslant s\leqslant t)$$ for every $$t\geqslant0$$.

This is not the definition of the Markov property. The Markov property is the assumption that, for every $$t\geqslant1$$, the conditional distribution of $$X_t$$ conditionally on $$\mathcal F_{t-1}$$ depends on $$X_{t-1}$$ only. Obviously, if the conditional distribution of $$X_t$$ conditionally on $$\mathcal F_{t-1}$$ is the conditional distribution of $$X_t$$ conditionally on $$X_{t-1}$$ (Markov), then the same is true for the conditional expectations (the property you think is Markov), but the converse is not true.

For a counterexample, assume that $$(Z_t)_{t\geqslant2}$$ is independent, integrable, centered, non constant (say, with a standard normal distribution), and independent on $$X_0$$. Let $$X_1=X_0=1$$ and $$X_t=X_{t-1}+Z_tX_{t-2}$$ for every $$t\geqslant2$$.

Then $$\mathrm E(X_t\mid \mathcal F_{t-1})=\mathrm E(X_t\mid X_{t-1})=X_{t-1}$$ for every $$t\geqslant1$$ (hence, if $$X_0$$ is integrable, $$(X_t)_{t\geqslant0}$$ is a martingale) but $$(X_t)_{t\geqslant0}$$ is not a Markov process since the conditional distribution of $$X_t$$ conditionally on $$\mathcal F_{t-1}$$ does not depend on $$X_{t-1}$$ only but on $$(X_{t-1},X_{t-2})$$.

• Thanks for the reply. Could you please show more details? I don't understand how to show $E(Z_tX_{t-2}|X_{t-1})=0$. I did use the definition of Markov property: $E(X_{t}|\mathcal{F}_{t-1})=E(X_{t}|\sigma(X_{t-1}))$. Following your own link of wiki, you can see this under "Alternative Formulations". Feb 26, 2012 at 23:44
• No, you forgot for all $f$ bounded and measurable. So I was right, this is really the point where there is a deep misconception in your view of Markovianity.
– Did
Feb 27, 2012 at 0:05
• Re your question about $E(Z_tX_{t-2}\mid X_{t-1})$, a general result you might wish to use is that if $U$ is independent on $(V,W)$, then $E(UV\mid W)=E(U)E(V\mid W)$.
– Did
Feb 27, 2012 at 0:10
• Thanks. Yes, that property is true for sure. However, $E(Z_tX_{t−2}∣X_{t−1})=Z_tE(X_{t−2}∣X_{t−1})=0$ ? I guess, since your assumption is about the $Z_t$ being symmetric Bernoulli, you would in the end use the expectation of $Z_t$ to get the zero. Feb 27, 2012 at 1:01
• By the way, rereading your comments, I wonder whether you would know how to prove the property you declared true for sure. In my opinion, to prove it in details would be highly enlightening.
– Did
Mar 2, 2012 at 12:48