Let $\{X_n\}$ be an irreducible Markov chain. Does exist example of such $\{X_n\}$ which is also a martingale given that:

a. $\{X_n\}$ is recurrent with finite number of states (but bigger than $1$)?

b. $\{X_n\}$ is recurrent with infinite states ?

c. $\{X_n\}$ is transient with infinite states ?

(If not give a proof why it cannot be)

In another question I saw that exists examples for Martingales which are not Markov chains and I know that given harmonic function $h$ on the markov chain, then $h(X_n)$ is indeed a martingale.

About c: I think about a random walk on $\mathbb{Z}$ defined by $$X_n=\cases{0 \quad p=\frac 1 2\\X_n+1\quad p=\frac 1 2}$$ which is Markov as a random walk but I don't know how to prove it's a martingle (I'm even pretty sure it's not).

About A I thought taking a series of absorbent states, e.g. $\left(\begin{array}{cc} 1 & 0\\ 0 & 1 \end{array}\right) $ but again I don't know how to prove it's martingle. About the second example (means b.) I'm clueless.

Are my examples correct? If so: How I prove marginality and find example for b.? If they are incorrect: Which examples can I find for these questions?


1 Answer 1


Note that for a Markov chain, the martingale condition $\mathbb E[X_{n+1}\mid X_n]$ is equivalent to $$\sum_{j\in S} jP_{ij} = i, \; \forall i\in S.\tag 1$$

For A, if $\{X_n\}$ is a martingale then in particular $$\mathbb E[X_{n+1}\mid X_n=1] = \sum_{j=1}^m jP_{1j}=1, $$ which implies that $P_{11}=1$, so $\{X_n\}$ cannot be irreducible. In fact, states $1$ and $m$ are absorbing, and the other states are transient.

For B, consider the symmetric random walk on $\mathbb Z$, which is a martingale and (null) recurrent.

For C, I am fairly sure the answer is "no," but am unsure how to prove it.

  • $\begingroup$ Could you give me a pointer on how to show that the in-between states are transient, please? My stochastic processes textbook asks me to show that. I'm trying to show that $P(x,0)$ or $P(x,d)$ are positive and thus $x\sim y$ where $y$ is absorbing. $\endgroup$ Oct 8, 2021 at 13:43

You must log in to answer this question.