Markov chains, invariant distributions and long-run behaviour I have some general questions concerning discrete Markov chains, their invariant distributions, and their long-run behaviour. From the research I have done on the internet, I understand that there are two planes of terminology that are used in this field, the language of independent variables (from probability theory) and the language of matrices (from linear algebra); and I am trying to understand how they fit together and actually say the same thing. Before formulating my precise questions, let me state the two definitions of Markov chains I have come across:

Let $X = (X_0, X_1, \dots $ be a sequence of random variables taking values in the state space $S$. We assume that $S$ is countable. We say that $X$ has the Markov property if for all $n \geq 0$, $i_0, \dots , i_{n+1} \in S$, we have $$ \mathbb{P}(X_{n+1}=i_{n+1} \mid X_0=i_0, \dots, X_n=i_n) = \mathbb{P}(X_{n+1}=i_{n+1} \mid X_n=i_n) .$$
If $X$ has the Markov property, we call it a Markov chain.

In a set of slides on Linear Algebra and its applications, I found the following definition (for which I suppose we assume that the state space is finite).

A probability vector or probability distribution is a vector $v\in\mathbb{R}^n$ with non-negative entries (probabilities) that add up to 1. A stochastic matrix $P$ is a square matrix whose columns are probability vectors. A Markov chain is a sequence of probability vectors $(x_k)_{k\in\mathbb{N}}$, together with a stochastic matrix $P$, such that $$x_k = P^k x_0 \text{ (or equivalently } x_k = P x_{k-1} \text{)} \text{ for all } k\in\mathbb{N} .$$ The vector $x_0$ is referred to as the initial state and a vector $x_k$ of a Markov chain is called a state vector.

Here are my questions:

*

*What is the difference between the two definitions? Why is it preferable to define Markov chains as a sequence of random variables, aside from allowing us to consider infinite state spaces? Would the former definition in that sense be a generalization of the latter?

*An invariant distribution or steady-state vector is a probability vector $ \pi \in \mathbb{R}^n$ such that $\pi = P\pi$, that is, an eigenvector of the transition matrix $P$ which at the same time qualifies as a probability vector. What are sufficient (and possibly necessary) conditions for a matrix $P$ to have such a particular eigenvector? In the set of notes from which I have drawn the first definition, I read that there exists a unique invariant distribution if the Markov chain is irreducible and positive recurrent. What does this condition mean when viewed through the optics of linear algebra?

*The set of notes I have just referred to goes on to say that for a Markov chain that is irreducible, positive recurrent and aperiodic, with transition matrix $P=(p_{i,k})$, we have that $p_{i,k}(n) \to \pi_k$ as $n\to\infty$, where $P^n = \big(p_{i,k}(n)\big)$ and $\pi_k$ is the $k$th component of the unique invariant distribution $\pi$. Again, I am wondering how this condition might be formulated in terms of matrix theory. When investigating the long-run behavior of a Markov chain, isn't the question at heart under which conditions the powers of a stochastic matrix $P^n$ converge to some limit matrix? In the slides from which I have drawn the second definitions, I have come across the following theorem and I am wondering whether it is equivalent to the result I have just stated above.


Let $P$ be an $n \times n$ stochastic matrix. Then we say that $P$ is regular if some matrix power $P^k$ contains no zero entries.
If $P$ be an $n \times n$ regular stochastic matrix, then $P$ has a unique steady-state vector $q$ that is a probability vector. Furthermore, if $x_0$ is any initial state and
$x_k=P^k x_k$ (or equivalently $x_k = P x_{k-1}$), then the Markov chain
$(x_k)_{k\in\mathbb{N}}$ converges to $q$ as $k\to\infty$.

By the way, in a high school textbook, I have come across a similar (equivalent?) condition for convergence:

Let $P$ be a stochastic matrix. Then the sequence $P^k$ converges (component-wise) to a limit matrix $M$ if the sequence $(P^n)_{\in\in\mathbb{N}}$ contains a matrix which in turn contains at least one row with non-zero entries. In this case, the columns of $M$ are all equal and any probability distribution $x$ satisfies the equation $\pi = Mx$, where $\pi$ is a solution to the equation $\pi = M\pi$.

I hope I have been sufficiently clear and would be very grateful for an answer to my questions (or else a recommendation of a textbook or lecture notes which explain the connection between the two perspectives). Many thanks for having read this up to the end! :)
 A: Regarding question 1:
Just the linear algebra view only allows you to describe the evolution of distributions and the evolution of expectations. It loses the notion of a realization of a path (which requires resolving the joint distribution of the process at various times).
Regarding question 2:
There's a fairly simple linear algebra way of looking at the invariant distribution question. First, it is automatic from the fact that the rows sum to $1$ that the column vector of all $1$s is an eigenvector with eigenvalue $1$. From linear algebra, $P$ and $P^T$ have the same eigenvalues, so an invariant distribution exists.
As for the irreducibility, you can look at it in the same way. If there is a proper subset of the state space such that if you start in that subset then at the next time step you will remain in that subset*, then the column vector of all $1$s on that subset and $0$s elsewhere is an eigenvector with eigenvalue $1$, which is linearly independent of the one we already found. Again linear algebra also tells you that the geometric multiplicities of the eigenvalues of $P$ and $P^T$ are the same, so in this situation the invariant distribution is not unique.
Note that irreducibility is not necessary for the invariant distribution to be unique, as you can see from the deterministic process $1 \to 2,2 \to 2$.
I don't think you will get much intuition about the positive recurrence requirement from linear algebra, because in finite state spaces irreducibility implies positive recurrence.
Regarding question 3:
In linear algebra language the matter of approaching a fixed invariant distribution regardless of the initial distribution comes down to whether $P^n$ has a limit with all rows equal. This is controlled by the eigenvalues which have modulus greater than or equal to 1: specifically it happens if there is at most one eigenvector whose eigenvalue has modulus greater than or equal to $1$, and if such an eigenvector exists then the eigenvalue is $1$.
Irreducibility guarantees that there is only one eigenvector with eigenvalue $1$ (though I did not really explain why here). Aperiodicity guarantees that there are no eigenvalues other than $1$ that have modulus $1$. Without this assumption you can have a failure to approach an invariant distribution, as demonstrated by the deterministic process $1 \to 2,2 \to 1$.
* I call chains satisfying this hypothesis "reducible", although this is not standard terminology. Unfortunately, there are chains that are neither "reducible" nor irreducible; these are chains whose reachability relation graph is weakly but not strongly connected.
A: There are a circle of results known as the Perron Frobenius theorem which describe positive matrices and include the results you mentioned. All the proofs can be found here:
https://mtaylor.web.unc.edu/wp-content/uploads/sites/16915/2018/04/linalg.pdf
in section 4.4 (page 181). Matrices are studied because the row-stochastic matrix $P$ representing the Markov chain satisfies $p_{ij} = P(X_1 = j | X_0 = i)$ and $(P^k)_{ij} = P(X_k = j | X_0 = i)$.
