# Proving that a certain Markov process is a Martingale.

Let $$\{X_t:t=0,1,2,\ldots \}$$ be a homogeneous Markov chain with state space $$\mathcal{S}=\{1,\ldots,n\}$$ and transition matrix $$p_{ij}=\binom{n}{j}\left(\frac{i}{n}\right)^j\left(\frac{n-i}{n}\right)^{n-j}\;\;\text{ for }i,j=1,\ldots,n.$$ I want to show that the process is a Martingale with respect to the natural filtration $$\{\mathcal{F}_t\}$$. So we compute $$E[X_{t+1}\mid \mathcal{F}_t](\omega)=E[X_{t+1}\mid \sigma(X_t)](\omega)=E[X_{t+1}\mid X_t]\circ X_t(\omega),$$ where $$E[X_{t+1}\mid X_t](j)=E[X_{t+1}\mid X_t=j],\;\;\;\;\text{ for all } j\in\mathcal{S}.$$ (I'm expecting $$E[X_{t+1}\mid X_t=j]$$ to be equal to $$j$$ for the process to be a Matringale). So I computed $$E[X_{t+1}\mid X_t=j]=\frac{E[1_{\{X_t=j\}}X_{t+1}]}{ \mathbb{P}\{X_t=j\}} = \frac{\sum_{i=1}^ni\cdot\mathbb{P}(X_{t+1}=i\mid X_t=j)}{\sum_{i=1}^n p_{ij} \mathbb{P}(X_0=i)}=\frac{\sum_{i=1}^n i\cdot p_{ij}}{\sum_{i=1}^n p_{ij} \mathbb{P}(X_0=i)}$$

And I got stuck on the last part. First of all, is it true that $$E[1_{\{X_t=j\}}X_{t+1}]=\sum_{i=1}^ni\cdot\mathbb{P}(X_{t+1}=i\mid X_t=j) ?$$

If yes how to proceed? I know I still need to replace $$p_{ij}$$ with it's appropriate value but the computation seems like a mess!

• Not to nitpick, but you have defined $p_{ij}$ for $i,j=1,\ldots,n$. Should this not be for $i,j=0,\ldots,n$, since $\mathcal S = \{0,\ldots,n\}$? Nov 22, 2019 at 23:28
• Actually I see a problem there in which $p_{0j}=0$ for all $j>0$ and $p_{00}=1$, so that $0$ is an absorbing state, and similarly $p_{nn}=1$ so that $n$ is an absorbing state as well. Nov 22, 2019 at 23:31
• @Math1000 it was a typo. Fixed! Nov 22, 2019 at 23:48
• @UserA After the update $\sum_{j}p_{ij}<1$
– user140541
Nov 22, 2019 at 23:55
• @d.k.o. Indeed because if $X_{n+1}\mid X_n = i$ is $\mathrm{Bin}(n,i/n)$ then we no longer have the probability for $0$ successes. Nov 23, 2019 at 0:47

I assume that $$p_{0j}=1\{j=0\}$$ and $$p_{nj}=1\{j=n\}$$. Then $$\mathsf{E}[X_{t+1}\mid X_t=0]=0$$, $$\mathsf{E}[X_{t+1}\mid X_t=n]=n$$, and for $$0, $$\mathsf{E}[X_{t+1}\mid X_t=i]=\sum_{j=1}^nj\cdot p_{ij}=n\cdot\frac{i}{n}=i$$ because $$X_{t+1}\mid X_t=i\sim \text{Bin}(n,i/n)$$.
• thank you but you did not answer my question! you got that $E[X_{t+1}\mid X_t=i]=\sum_{j=1}^n j\cdot p_{ij}$ while I got that $E[X_{t+1}\mid X_t=i]=(\sum_{j=1}^n j\cdot p_{ij})/{\Pr\{X_t=i\}}$. What did I do wrong? Nov 23, 2019 at 0:19
• By definition $p_{ij}=\mathsf{P}(X_{t+1}=j\mid X_t=i)$.
• Then $\mathsf{E}[X_{t+1}\mid X_t=i]=\sum_{j=0}^n j\,\mathsf{P}(X_{t+1}=j\mid X_t=i)=\sum_{j=1}^n j\,p_{ij}$.