# Invariant Distribution of a Markov Chain

A Markov chain with transition matrix

\begin{align} P = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \end{align}

has a continuum of stationary distributions

$$\gamma \begin{bmatrix} 0 \\ 1 \end{bmatrix} + (1 - \gamma) \begin{bmatrix} 1 \\ 0 \end{bmatrix}; \; \; \forall \gamma \in [0,1]$$

And invariant functions

$$\begin{bmatrix} 0 \\ \alpha_1 \end{bmatrix} \; and \; \begin{bmatrix} \alpha_2 \\ 0 \end{bmatrix}; \; \; \alpha_1, \alpha_2 \in \mathbb{R}$$

However the book states the process is not ergodic when $\gamma \in (0,1)$, as neither invariant function is constant across states that receive positive probability according to a stationary distribution associated with $\gamma \in (0,1)$. Why is this? If $\gamma = .5$ then the Markov process has equal chance of starting in state $1$ or state $2$, but will stay there forever, hence the random variable is constant and should be invariant.

The wikipedia page on Markov chains states that "A Markov chain is ergodic if there is a number $N$ such that any state can be reached from any other state in exactly $N$ steps." Here the reachable states depend on the starting state, therefore the chain is not ergodic.
In the case where $0<\gamma<1$ the chain will indeed remain in the state it started with, but that initial state remains a random variable. If $X_1$ is the first state (a true r.v.), then the $k$-th state $X_k=X_1$ is obviously also a random variable. The ("deterministic") fact that $P(X_k=i|X_1=i)=1$ does not change that. The chain only looks deterministic if you assume that the first state is known (i.e., look at a conditional probability), but in fact it is not.
• Hi @A.G. Thanks for the answer; why are the invariant functions not constant when $\gamma \in (0,1)$, this is more of where my confusion lies – Sunhwa Nov 11 '16 at 16:55