Let $p \in [0, 1]$, consider a stochastic process $(X_n)_{n\in\mathbb{N}_0}$ with $X_0 = x_0 \in [0, 1]$ and the following dynamics: For $n\in \Bbb{N}_0$, conditional on $X_0, X_1, \ldots, X_n$, we have: $$X_{n+1}=\begin{cases} 1- p + p X_n & \text{with probability } X_n \\ p X_n & \text{with probability }1- X_n \end{cases}$$ Check that $(X_n)$ is a martingale and converges a. s. What is the law of its limit?

Since $(X_n)_{n\in\mathbb{N}_0}$ is a stochastic process with index set $\mathbb{N}_0$, we only have to show:$$\mathbf{E}[X_{n+1} | \mathcal{F}_n] = X_n \quad \forall n \in\mathbb{N}_0\, .$$ If we know $\mathcal{F}_n$, we also know $X_n$ and can easily calculate the conditional expectation:

\begin{align*} \mathbf{E}[X_{n+1} | \mathcal{F}_n] &= (1- p +p X_n)\cdot X_n + p X_n \cdot (1- X_n) \\ &= X_n -p X_n + p X_n^2 + p X_n - p X_n^2 \\& = X_n\, . \end{align*} so $(X_n)_{n\in\mathbb{N}_0}$ is a martingale.

Since $(X_n)_{n\in\mathbb{N}_0}$ is bounded by 1 it is uniformly integrable: $$\sup_{n\in\mathbb{N}_0} \int_{\{|X_n| > 1\}} |X_n| \, d\mathbf{P} = 0\,$$ and we can use the convergence theorem for uniformly integrable martingales: $X_n \rightarrow X_\infty$ a. s. and $$\mathbf{E}[X_0] = \mathbf{E}[X_\infty|\mathcal{F}_0]\, .$$

Let $0 < a < b < 1$. If $\mathbf{P}\bigl[\{a < X_\infty < b\}\bigr] > 0$, $(X_n)_{n\in\mathbb{N}_0}$ must have at least one limit point $a \leq c \leq b$.

Let $0 \leq p <1$, assume that $X_n \rightarrow c$ on a set with non-zero probability.

Let $A:=\{|X_n-c| < \varepsilon\}$ for a certain $n\in\mathbb{N}_0$, we show that if $$\varepsilon < \frac{1-p}{1+p}\min(c, 1-c)\,$$ then $\mathbf{P}\bigl[\{|X_{n+1} - c| > \varepsilon\}\bigr | A] = 1$.

Assume that $1-c < c$ (or equivalently $c> 0.5$) and take any $\omega \in A$ for which $X_{n+1}(\omega) > X_n(\omega)$ (this happens with probability greater zero).

Obviously $X_n(\omega) > c - \varepsilon$, so \begin{align*} X_{n+1}(\omega) - X_n(\omega) &= 1- p - (1 -p) \, X_n(\omega) > 1- p - (1-p) (c-\varepsilon) \\&= (1-p) (1-c +\varepsilon) \\&= (1-p)\Bigl((1-c) +\frac{1-p}{1+p} (1-c)\Bigr) \\&= (1-p)(1-c)\Bigl(1+\frac{1-p}{1+p}\Bigr) \\&= 2 \cdot \frac{1-p}{1+p}(1-c) = 2 \varepsilon \, . \end{align*} Similarly for an $\omega \in A$ with $X_{n+1}(\omega) < X_n(\omega)$ and also for the case that $1-c\geq c$.

This means $$|X_n-c| < \varepsilon \Longrightarrow |X_{n+1}-c| > \varepsilon \, .$$

So for any $0 < a < b < 1$ we have $\mathbf{P}\bigl[\{a < X_\infty < b\}\bigr] = 0$. But $\mathbf{E}[X_\infty]= \mathbf{E}[X_0] = x_0$, so the law of $X_\infty$ must be: $$\mathbf{P}[X_\infty = 1] = x_0, \quad \mathbf{P}[X_\infty = 0] = 1-x_0 \, .$$

In the case $p = 1$, we directly see that $$\mathbf{P}[X_\infty = x_0] = 1 \, .$$

Could you please check, if my solution is correct? Thanks in advance. I had trouble to write it down mathematically clean without becoming very verbose.

• Computing $$E(X_{n+1}(1-X_{n+1})|\mathcal F_n)=p(2-p)X_n(1-X_n)$$ yields the result right away.
– Did
Apr 7, 2015 at 20:14

@Did's solution is the slick approach to proving $X_\infty$ equals $0$ or $1$ when $p<1$. But your approach is also valid, after some modification. There are some errors in your solution:
1. The limit point $c$ depends on $\omega$. Given that $P(a < X_\infty < b) > 0$, it does not follow that there exists $c\in(a,b)$ such that $P(X_n\to c) > 0$ (e.g., consider if $X_\infty$ is uniformly distributed on $[0,1]$)
2. The inequality $1-p-(1-p)X_n(\omega) > 1-p-(1-p)(c-\varepsilon)$ is incorrect, since it doesn't follow from $X_n > c-\varepsilon$. You want $X_n < c+\varepsilon$, which means your definition of $\varepsilon$ needs to be altered.
3. $P(X_n\to c)=0$ for every $c\in(a,b)$ does not imply that $P(X_\infty\in(a,b))=0$. Consider the case of $X_\infty$ uniform.
But the proof can be salvaged. You need to argue pointwise, as you've done. All you need to show (if $p<1$) is that for every $\omega$ and $c\in(0,1)$ there exists $\varepsilon > 0$ such that if $|X_n(\omega)-c| < \varepsilon$ then $|X_{n+1}(\omega)-c| > \varepsilon$. Therefore for every $\omega$ such that $X_n(\omega)\to X_\infty(\omega)$, either $X_\infty(\omega)=0$ or $X_\infty(\omega)=1$ and so $$P(X_\infty=0 {\rm\ or\ } X_\infty=1) = 1 \,.$$