Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

1. On the internet, it is suggested that $$ X_t=\left(\int_0^t X_s \;ds\right)\;dW_{t} $$ is a martingale, but not a Markov process. I understand that the process $$ I_t(C)=\int_0^t C_s \; dW_s$$ is a martingale, but I don't see $X_t$ as an Ito's integral. In order to show it is a martingale but not a Markov process, I want to show that $$ E(X_t\mid\mathcal{F}_{t-1})=X_{t-1}\neq E(X_t\mid\sigma(X_{t-1})). $$ I don't know how to take the expecation in this case.

Other counterexamples available online might be wrong. (Or, I could be wrong.)

2. Invalid counterexample: Let $e(t)$ be a random variable which is not independent of $y(0)$ and $$ y(t)=y(t-1)+e(t)-E{[}e(t)|F(t-1){]}. $$ Then y(t) is a martingale but it is not a Markov process.

I will show that it is both a martingale and a Markov process. Define $X_{t}$ as $$ X_{t}=X_{t-1}+e_{t}-E\left(e_{t}|\mathcal{F}_{t-1}\right) $$

Then, $\{X_{t}\}$ is a martingale because $$ E(X_{t}|\mathcal{F}_{t-1}) = E\left(X_{t-1}+e_{t}-E\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\mathcal{F}_{t-1}\right) = E(X_{t-1}|\mathcal{F}_{t-1})+E\left(e_{t}|\mathcal{F}_{t-1}\right)-E\left(e_{t}|\mathcal{F}_{t-1}\right) = E(X_{t-1}|\mathcal{F}_{t-1}) = X_{t-1} $$

However, it is also a Markov process. $$ E(X_{t}|\sigma(X_{t-1})) = E\left(X_{t-1}+e_{t}-E\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\sigma(X_{t-1})\right) = E(X_{t-1}|\sigma(X_{t-1}))+E\left(e_{t}|\sigma(X_{t-1})\right)-E\left(E\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\sigma(X_{t-1})\right) $$

Since $\sigma(X_{t-1})\subset\mathcal{F}_{t-1}$, by the towering property, $$ E\left(\left(e_{t}|\mathcal{F}_{t-1}\right)\bigg|\sigma(X_{t-1})\right)=E\left(e_{t}|\sigma(X_{t-1})\right). $$

Thus $$E(X_{t}|\sigma(X_{t-1})) = E(X_{t-1}|\sigma(X_{t-1}))+E\left(e_{t}|\sigma(X_{t-1})\right)-E\left(e_{t}|\sigma(X_{t-1})\right) = E(X_{t-1}|\sigma(X_{t-1}) $$

Since $X_{t-1}$ is $\sigma(X_{t-1})$-measurable, $E(X_{t-1}|\sigma(X_{t-1}))=X_{t-1}$. Thus $$ E(X_{t}|\sigma(X_{t-1}))=X_{t-1}=E(X_{t}|\mathcal{F}_{t-1}) $$ Hence $\{X_{t}\}$ is also a Markov process.

3. Invalid counterexample from the textbook answer key. Alison Etheridge's textbook gives an example: $$ S_{n+1}=S_{n}+\xi_{n+1} $$ where $\xi_{n}\in\{-1,1\}$ and $\{\xi_{n}\}_{n\geq0}$ are iid. Then, $$ P\left(S_{n+1}=k+1\bigg|S_{n}=k\right)=p \text{and }P\left(S_{n+1}=k-1\bigg|S_{n}=k\right)=1-p. $$ The author claims that $\left\{ S_{n}\right\} $ is a Markov process.

Then in the homework answer key, someone gives $\left\{ Z_{n}\right\}$ as a counterexample of a martingale not being a Markov process: $ \displaystyle{Z_{n}=\sum_{i=1}^{n}\xi_{i}} $ where $P(\xi_{i}=1)=\frac{1}{2}=P(\xi_{i}=-1)$.

Obviously, $Z_{n+1}=Z_{n}+\xi_{n+1}.$ So what's the difference between this counterexample and the example? Moreover, given $P(\xi_{i}=1)=\frac{1}{2}=P(\xi_{i}=-1)$, I think is both $\left\{ Z_{n}\right\}$ is both a martingale and a Markov process.

share|cite|improve this question
1. the definition of $X$ is incorrect - it seems that there should be $dX_t$. Could you give a link for this example? $$ $$ 2. and 3. - I think you are right and your calculations are correct (in 2. I assume that $\mathcal F_t$ is a natural filtration of $y$ or $X$ in your notation). In 3. you have a random walk which is a Markov process, and a martingale if its jump has a zero mean. – Ilya Feb 26 '12 at 22:22
@Ilya, Thanks for the editing. The link is here – user16859 Feb 26 '12 at 22:36
Here is another example I mentioned:… I need a valid counterexample. Thank you! – user16859 Feb 26 '12 at 22:39
up vote 12 down vote accepted

There is a pervasive mistake in your post, possibly explaining your trouble, which is to believe that $(X_t)_{t\geqslant0}$ being a Markov process means that $\mathrm E(X_t\mid \mathcal F_{t-1})=\mathrm E(X_t\mid X_{t-1})$ for every $t\geqslant1$, where $\mathcal F_t=\sigma(X_s;0\leqslant s\leqslant t)$ for every $t\geqslant0$.

This is not the definition of the Markov property. The Markov property is the assumption that, for every $t\geqslant1$, the conditional distribution of $X_t$ conditionally on $\mathcal F_{t-1}$ depends on $X_{t-1}$ only. Obviously, if the conditional distribution of $X_t$ conditionally on $\mathcal F_{t-1}$ is the conditional distribution of $X_t$ conditionally on $X_{t-1}$ (Markov), then the same is true for the conditional expectations (the property you think is Markov), but the converse is not true.

For a counterexample, assume that $(Z_t)_{t\geqslant2}$ is independent, integrable and centered (say, with a normal distribution), and independent on $X_0$. Let $X_1=X_0=1$ and $X_t=X_{t-1}+Z_tX_{t-2}$ for every $t\geqslant2$.

Then $\mathrm E(X_t\mid \mathcal F_{t-1})=\mathrm E(X_t\mid X_{t-1})=X_{t-1}$ for every $t\geqslant1$ (hence, if $X_0$ is integrable, $(X_t)_{t\geqslant0}$ is a martingale) but $(X_t)_{t\geqslant0}$ is not a Markov process since the conditional distribution of $X_t$ conditionally on $\mathcal F_{t-1}$ does not depend on $X_{t-1}$ only but on $(X_{t-1},X_{t-2})$.

share|cite|improve this answer
Thanks for the reply. Could you please show more details? I don't understand how to show $E(Z_tX_{t-2}|X_{t-1})=0$. I did use the definition of Markov property: $E(X_{t}|\mathcal{F}_{t-1})=E(X_{t}|\sigma(X_{t-1}))$. Following your own link of wiki, you can see this under "Alternative Formulations". – user16859 Feb 26 '12 at 23:44
No, you forgot for all $f$ bounded and measurable. So I was right, this is really the point where there is a deep misconception in your view of Markovianity. – Did Feb 27 '12 at 0:05
Re your question about $E(Z_tX_{t-2}\mid X_{t-1})$, a general result you might wish to use is that if $U$ is independent on $(V,W)$, then $E(UV\mid W)=E(U)E(V\mid W)$. – Did Feb 27 '12 at 0:10
Thanks. Yes, that property is true for sure. However, $E(Z_tX_{t−2}∣X_{t−1})=Z_tE(X_{t−2}∣X_{t−1})=0$ ? I guess, since your assumption is about the $Z_t$ being symmetric Bernoulli, you would in the end use the expectation of $Z_t$ to get the zero. – user16859 Feb 27 '12 at 1:01
Please read my comment again: the RHS has a factor E(U), not U. – Did Feb 27 '12 at 1:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.