When does a Markov chain not have a steady state? I was asked this question on an oral qual, and eventually I seemed to conclude that there have to be eigenvalues of modulus 1.  But I just realised every Markov matrix has one as an eigenvalue.  So should the condition be multiple eigenvalues with modulus 1?
 A: You are correct. If $P$ is a Markov transition matrix, then $P^n$ converges 
as $n\to\infty$ if and only if $P$'s only eigenvalue of modulus one is $\lambda=1.$ 
A: Every Markov chain on a finite state space has an invariant distribution. As you said, this follows directly from the condition that the rows sum to $1$.
It is possible for a Markov chain on a finite state space to have multiple invariant distributions. However, the Perron-Frobenius theorem tells us that these can be decomposed into distributions which are concentrated on strongly connected components of the state space. Decomposing the process into strongly connected components results in one or more different Markov chains each of which has a unique invariant distribution.
Any transient states of the original chain will not be states in any of these sub-chains, since they are not in any strongly connected component. Confusingly, this means that a chain that is not irreducible is generically not reducible into chains on disjoint subsets of the state space. That's because generically some transient states would have to be in more than one sub-chain, which is because generically one can reach more than one strongly connected component starting from a transient state.
As an important special case, an irreducible finite state Markov chain has a unique invariant distribution which assigns positive probability to all states.
However many invariant distributions the process has, it can happen that no invariant distribution is approached over time. When the state space is finite, it turns out that this only happens when the chain is "periodic" (meaning that there are states $i,j$ and an integer $n>1$ such that all paths from $i$ to $j$ have a length which is a multiple of $n$). In this case the transition matrix has an eigenvalue which is not $1$ and has modulus $1$. If the corresponding eigenvector contributes to the initial condition, then its contribution does not decay, and no invariant distribution is approached. The classic example of a periodic chain is $P=\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$, but in general it is not essential that the chain is deterministic for it to be periodic.
It is possible for a Markov chain on an infinite state space to not have any invariant distributions. This is roughly because probability mass can escape to infinity. A simple example is a Markov chain on $\mathbb{Z}$ which deterministically moves one unit to the right at every step. Another example is the simple symmetric random walk: it has an invariant measure which is uniform on $\mathbb{Z}$, but this measure cannot be normalized.
