Markov Chain Expectation Variance Covariance Given a Markov chain on {0, 1} defined by:
\begin{gather}
    P = \begin{bmatrix} 1-p & p \\ q & 1-q  \end{bmatrix} 
\end{gather}
With an initial distribution $\pi(0)$ = [$\frac{q}{p+q}$,$\frac{p}{p+q}$]. I have to find the Expectation $\mathbf{E}[X_n]$, the Variance $\mathbf{V}[X_n]$ and the $\mathbf{Cov}[X_{m+n}, X_n]$. 
Intuitively I would say that I can compute the expectation and the variance by multiplying {0,1} with the stationary probabilities. I this correct?  How would I factor in the covariance? Isn't this just equal to 0, because the chain converges to a stable distribution i.e. there is no difference between time m=n and n in the long run?
 A: Note that because $\mathbb P(X_n\in\{0,1\})=1$ we have $\mathbb E[X_n] = \mathbb P(X_n=1)$ for all $n\geqslant 1$. Now, \begin{align}\mathbb P(X_n=1) &= \mathbb P(X_n=1\mid X_0=0)\mathbb P(X_0=0) + \mathbb P(X_n=1\mid X_0=1)\mathbb P(X_0=1)\\ &=
\mathbb P(X_n=1\mid X_0=0)\frac q{p+q} + \mathbb P(X_n=1\mid X_0=1) \frac p{p+q}
\end{align}
Computing $P^n$ we have
$$
P^n = \left(
\begin{array}{cc}
 \frac{p (-p-q+1)^n+q}{p+q} & \frac{p-p (-p-q+1)^n}{p+q} \\
 \frac{q-(-p-q+1)^n q}{p+q} & \frac{q (-p-q+1)^n+p}{p+q} \\
\end{array}
\right),
$$
so that 
$$
\mathbb P(X_n=1\mid X_0=0) = \frac{p-p(1-p-q)^n}{p+q}
$$
and
$$
\mathbb P(X_n=1\mid X_0=1) = \frac{q(1-p-q)^n+p}{p+q},
$$
so that
\begin{align}
\mathbb E[X_n] &= \mathbb P(X_n=1)\\
&= \frac{p-p(1-p-q)^n}{p+q}\cdot \frac q{p+q} + \frac{q(1-p-q)^n+p}{p+q}\cdot \frac p{p+q}\\
&=\frac p{p+q}.
\end{align}
Note that this is just $\mathbb P(X_0=1)$. Similarly, we may compute the variance of $X_n$ by the variance of $\pi$:
$$
\mathrm{Var}(X_n) = \mathrm{Var}(\pi) = 1^2\mathbb P(X_0=1) = \frac p{p+q}.
$$
As for the covariance of $X_{m+n}$ and $X_n$, because the process is stationary, this reduces to the covariance of $X_m$ and $X_0$,
which is
$$
\mathrm{Cov}(X_m,X_0) = \mathbb E[X_mX_0] - \mathbb E[X_m]\mathbb E[X_0] = \mathbb E[X_mX_0] - \left(\frac p{p+q}\right)^2.
$$
Now,
\begin{align}
\mathbb E[X_mX_0] &= \mathbb E[X_mX_0\mid X_0=0]\mathbb P(X_0=0) + \mathbb E[X_mX_0\mid X_0=1]\mathbb P(X_0=1)\\ &= \mathbb E[X_m]\mathbb P(X_0=1)\\
&= \left(\frac p{p+q}\right)^2,
\end{align}
so that $\mathrm{Cov}(X_m,X_0)=0$.
