Markov Chain Exercise $2$ step I'm studying for myself Markov Chain. I'm doing a particular exercise:
Consider the time series whose 2-step conditional probability is 
$$ X_t \sim 
\begin{cases}
Ber(0.90) & \mbox{if}\quad X_{t-1}=X_{t-2}=1; \\
Ber(0.10) & \mbox{if}\quad X_{t-1}=X_{t-2}=0; \\
Ber(0.50) & \mbox{otherwise.} \\
\end{cases}
$$
Found the Unconditionally distribution of $X_t$. I know that the solution is $X_t \sim Ber(0.5)$.
The Hint that the book gives me is:
Consider the initial conditional $X_0=X_{-1}=0$. Introduce the vector with four components $\textbf{X}_t$ where 


*

*$(1,0,0,0)'$ correspond at the state $X_t=X_{t-1}=0$

*$(0,1,0,0)'$ correspond at the state $X_t=0,X_{t-1}=1$

*$(0,0,1,0)'$ correspond at the state $X_t=1,X_{t-1}=0$

*$(0,0,0,1)'$ correspond at the state $X_t=X_{t-1}=1$


Introduce the $4 \times 4$ transiction matrix from $\textbf{X}_0$ to $\textbf{X}_1$, express the state $\textbf{X}_k$ in function of initial state e transiction matrix, consider the limit $k \to \infty$ and take a conclusion.
My work:
I think that transiction matrix is 
$$P=\begin{pmatrix}
0.9 & 0 & 0.1 & 0 \\
0.5 & 0 & 0.5 & 0 \\
0 & 0.5 & 0 & 0.5 \\
0 & 0.1 & 0 & 0.9 \\
\end{pmatrix}
$$
I don't know to express how to express $\textbf{X}_k$ in function of initial state (that i suppose is $(1,0,0,0)'$). I found the stationary distribution $\pi$ using
$$\pi P=\pi$$
that is $$\pi=(5/12,1/12,1/12,5/12)$$
but this not give me any information.
Could anyone help me please? I'm very confused. Thank you all. Sorry for my bad english.
 A: As I understand it, you are looking to show that in the limit $t \rightarrow \infty$, $X_t$ converges in distribution to $\text{Ber}(1/2)$. This is equivalent to proving:
$$\lim_{t \rightarrow \infty} P(X_t = 1) = \frac{1}{2}.$$
Let $Y_t = (X_t, X_{t-1})$ denote the Markov chain that is described in the original question; the transition matrix $P$, and stationary distribution $\pi$ have already been calculated in the question.
We will proceed with two steps.


*

*Prove that the Markov chain $Y_t$ has a limiting distribution, and that this is given by $\pi$.

*Use this to obtain the limiting distribution for $X_t$.


Note that Step 1 is not trivial, because Markov chains do not neccessarily have a limiting distribution, even when they have a stationary distribution. First I prove Step 2, and come back to Step 1. 
Proof of 2.
Assuming that we have proven step 2, then for $(x,y) \in \{0,1\} \times \{0,1\}$
\begin{align*}
\lim_{t \rightarrow \infty} P\big(Y_t = (x,y) \big) &= \lim_{t \rightarrow \infty} P\big((X_{t-1}, X_t) = (x,y) \big)  \\
& = \pi(x,y)
\end{align*}
Hence
\begin{align*}
\lim_{t\rightarrow \infty} P(X_t = 1) & = 
\lim_{t \rightarrow \infty} P\big(\{Y_t = (0,1)\} \cup \{Y_t = (1,1)\} \big) \\
& = \lim_{t \rightarrow \infty} \bigg( P\big( Y_t = (0,1) \big) + P(\big(Y_t = (1,1) \big) \bigg) \\
& = \lim_{t \rightarrow \infty} P\big(Y_t = (0,1) \big) + \lim_{t \rightarrow \infty} P\big( Y_t = (1,1) \big) \\
& = \pi(0,1) + \pi(1,1) \\
& = \frac{1}{12} + \frac{5}{12} \\
& = \frac12,
\end{align*}
where the final lines followed from the formula given for $\pi$ in the question. 
Proof of 1.
We now return to show that $\lim_{t\rightarrow \infty} P\big(Y_t = (x,y) \big) = \pi(x,y)$.
Since the state space of $Y_t$ is finite (there are only four states), it is sufficient to show that $Y_t$ is an aperiodic Markov chain.
In particular, if we can show that there is a $k \geq 0$ such that all entries of $P^k > 0$, then this ensures aperiodicity. In fact, if we set $k = 2$ we see
$$P^2
=
\begin{pmatrix}
0.9 & 0 & 0.1 & 0 \\
0.5 & 0 & 0.5 & 0 \\
0 & 0.5 & 0 & 0.5 \\
0 & 0.1 & 0 & 0.9 \\
\end{pmatrix}^2
=
\begin{pmatrix}
0.01 & 0.45 & 0.09 & 0.45 \\
0.05 & 0.25 & 0.45 & 0.25 \\
0.25 & 0.05 & 0.25 & 0.45 \\
0.05 & 0.09 & 0.05 & 0.81 \\
\end{pmatrix},
$$
since all entries are greater than $0$, we are done.
