Exercise about martingale convergence 
Let $p \in [0, 1]$, consider a stochastic process $(X_n)_{n\in\mathbb{N}_0}$ with $X_0 = x_0 \in [0, 1]$ and the following dynamics: For $n\in \Bbb{N}_0$, conditional on $X_0, X_1, \ldots, X_n$, we have:
  $$ X_{n+1}=\begin{cases} 1- p + p X_n & \text{with probability } X_n \\
p X_n & \text{with probability }1- X_n \end{cases}$$
  Check that $(X_n)$ is a martingale and converges a. s. What is the law of its limit?

Since $(X_n)_{n\in\mathbb{N}_0}$ is a stochastic process with index set $\mathbb{N}_0$, we only have to show:$$\mathbf{E}[X_{n+1} | \mathcal{F}_n] = X_n \quad \forall n \in\mathbb{N}_0\, .$$
If we know $\mathcal{F}_n$, we also know $X_n$ and can easily calculate the conditional expectation:
\begin{align*}
\mathbf{E}[X_{n+1} | \mathcal{F}_n] &= (1- p +p X_n)\cdot X_n + p X_n \cdot (1- X_n) \\ &= X_n -p X_n + p X_n^2 + p X_n - p X_n^2 \\& = X_n\, .
\end{align*}
so $(X_n)_{n\in\mathbb{N}_0}$ is a martingale.
Since $(X_n)_{n\in\mathbb{N}_0}$ is bounded by 1 it is uniformly integrable:
$$\sup_{n\in\mathbb{N}_0} \int_{\{|X_n| > 1\}} |X_n| \, d\mathbf{P} = 0\, $$
and we can use the convergence theorem for uniformly integrable martingales: $X_n \rightarrow X_\infty$ a. s. and 
$$\mathbf{E}[X_0] = \mathbf{E}[X_\infty|\mathcal{F}_0]\, .$$
Let $0 < a < b < 1$. If $\mathbf{P}\bigl[\{a < X_\infty < b\}\bigr] > 0$, $(X_n)_{n\in\mathbb{N}_0}$ must have at least one limit point $a \leq c \leq b$.
Let $0 \leq p <1$, assume that $X_n \rightarrow c$ on a set with non-zero probability.
Let $A:=\{|X_n-c| < \varepsilon\}$ for a certain $n\in\mathbb{N}_0$, we show that if $$\varepsilon < \frac{1-p}{1+p}\min(c, 1-c)\,$$ then $\mathbf{P}\bigl[\{|X_{n+1} - c| > \varepsilon\}\bigr | A] = 1$.
Assume that $1-c < c$ (or equivalently $c> 0.5$) and take any $\omega \in A$ for which $X_{n+1}(\omega) > X_n(\omega)$ (this happens with probability greater zero).
Obviously $X_n(\omega) > c - \varepsilon$, so
\begin{align*}
X_{n+1}(\omega) - X_n(\omega) &= 1- p - (1 -p) \, X_n(\omega) > 1- p - (1-p) (c-\varepsilon) \\&= (1-p) (1-c +\varepsilon) \\&= (1-p)\Bigl((1-c) +\frac{1-p}{1+p} (1-c)\Bigr) \\&= (1-p)(1-c)\Bigl(1+\frac{1-p}{1+p}\Bigr) \\&= 2 \cdot \frac{1-p}{1+p}(1-c) = 2 \varepsilon \, .
\end{align*} Similarly for an $\omega \in A$ with $X_{n+1}(\omega) < X_n(\omega)$ and also for the case that $1-c\geq c$.
This means $$|X_n-c| < \varepsilon \Longrightarrow |X_{n+1}-c| > \varepsilon \, .$$
So for any $0 < a < b < 1$ we have $\mathbf{P}\bigl[\{a < X_\infty < b\}\bigr] = 0$. But $\mathbf{E}[X_\infty]= \mathbf{E}[X_0] = x_0$, so the law of $X_\infty$ must be:
$$\mathbf{P}[X_\infty = 1] = x_0, \quad \mathbf{P}[X_\infty = 0] = 1-x_0 \, .$$
In the case $p = 1$, we directly see that 
$$\mathbf{P}[X_\infty = x_0] = 1 \, .$$
Could you please check, if my solution is correct? Thanks in advance. I had trouble to write it down mathematically clean without becoming very verbose.
 A: @Did's solution is the slick approach to proving $X_\infty$ equals $0$ or $1$ when $p<1$. But your approach is also valid, after
some modification. There are some errors in your solution:


*

*The limit point $c$ depends on $\omega$. Given that $P(a < X_\infty < b) > 0$,
it does not follow that there exists $c\in(a,b)$ such that $P(X_n\to c) > 0$ (e.g., consider if $X_\infty$ is uniformly distributed on $[0,1]$)

*The inequality
$1-p-(1-p)X_n(\omega) > 1-p-(1-p)(c-\varepsilon)$
is incorrect, since it doesn't follow from $X_n > c-\varepsilon$. You want $X_n < c+\varepsilon$, which means your definition of $\varepsilon$ needs to be  altered.

*$P(X_n\to c)=0$ for every $c\in(a,b)$ does not imply  that $P(X_\infty\in(a,b))=0$. Consider the case of $X_\infty$ uniform.
But the proof can be salvaged. You need to argue pointwise, as you've done. All you need to show (if $p<1$) is that for every $\omega$ and $c\in(0,1)$ there exists $\varepsilon > 0$ such that if $ |X_n(\omega)-c| < \varepsilon$ then $|X_{n+1}(\omega)-c| > \varepsilon $. Therefore for every $\omega$ such that $X_n(\omega)\to X_\infty(\omega)$, either $X_\infty(\omega)=0$ or $X_\infty(\omega)=1$ and so
$$P(X_\infty=0 {\rm\  or\  } X_\infty=1) = 1 \,.$$
