Prove or disprove: $A=\left\{P^k\mid k\in\mathbb{N}\right\}$ has a finite number of elements Let $P\in[0,1]^{n\times n}$  be a stochastic matrix, meaning for every $i\in[1,n]_{\mathbb{N}}$:
$$\sum_{j=1}^nP_{ij}=1$$
It is given that $P$ is not diagonalizable. Define the group $A$ to be a group of square $n\times n$ matrices such that:
$$A=\left\{P^k\mid k\in\mathbb{N}\right\}$$
Prove or disprove: $A$ has a finite number of elements.

I'm asking this question because it is often required to compute $P^n$ when dealing with Markov Chains. I did a little research and found out that a stochastic matrix doesn't have to be diagonalizable, what eventually led me to think - How can I compute $P^n$ if $P$ is not diagonalizable? The statement that I'm asking your help proving or disproving is somewhat related - since if it is true, then my problem is solved. It might be false, though. If it is indeed false - first, I would be glad to see why, and additionally, I'd be happy to hear your thoughts on what should I do if I am presented with a non-diagonalizable $P$ and need to compute $P^n$. Thanks!
 A: Here is a convenient counterexample: for $0 < \lambda < 1$ and $\alpha > 0$ sufficiently small (in particular $\alpha < \frac{1 - \lambda}{6}$), the matrix
$$
P = \frac{1 - \lambda}{3} \pmatrix{1\\1\\1}\pmatrix{1&1&1} + 
\alpha \pmatrix{1\\1\\-2}\pmatrix{1&-1&0} + \lambda I 
$$
is (doubly) stochastic with positive entries, is not diagonalizable, and has eigenvalues $1$ with multiplicity $1$ and $\lambda$ with multiplicity $2$ ($I$ denotes the identity matrix of size $3$). Because $P$ has an eigenvalue of magnitude not equal to $1$ or $0$, $\{P^k\mid k \in \Bbb N\}$ will contain infinitely many elements.

In this case, there was no reason to go about computing $P^n$ explicitly: if $P$  has an eigenvalue $\lambda$ with $|\lambda|\notin \{0,1\}$, then it necessarily holds that $\{P^k:k \in \Bbb N\}$ contains infinitely many elements. Indeed: suppose for the purpose of contradiction that there are finitely many elements. By the pigeonhole principle, there exist integers $k_1 \leq k_2$ for which $P^{k_1} = P^{k_2}$. If $v$ is the eigenvector associated with $\lambda$, then we have
$$
P^{k_1}v = P^{k_2}v \implies \lambda^{k_1}v = \lambda^{k_2}v \implies \lambda^{k_1 - k_2} = 1 \implies |\lambda|^{k_1 - k_2} = 1.
$$
However, this would imply that $|\lambda| = 1$, contradicting our premise.
For the matrix constructed above, note that the vector $v = (1,1,-2)^T$ satisfies $Pv = \lambda v$.

If you are interested in computing $P^n$ nevertheless, then we can make use of the fact that $P$ can be written in the form $P = D + N$ where $D$ is diagonalizable, $N^2 = 0$, and $DN = ND$ (this is a "Jordan-Chevalley" decomposition). In particular, we have
$$
D = \frac{1 - \lambda}{3} \pmatrix{1\\1\\1}\pmatrix{1&1&1}+ \lambda I, \\
N = \alpha \pmatrix{1\\1\\-2}\pmatrix{1&-1&0}.
$$
Verify that we have
$$
D^n = \frac{(1 - \lambda)^n}{3} \pmatrix{1\\1\\1}\pmatrix{1&1&1}+ \lambda^n I.
$$
From there, use the binomial theorem to compute
$$
P^n = (D+ N)^n = D^n + nD^{n-1}N + 0 = D^n + nD^{n-1}N\\
= D^n + n\lambda^{n-1}N.
$$

Another way to write $P$ (for ease of presentation):
$$
P = \lambda I + \pmatrix{1&1\\1&1\\1&-2} \pmatrix{(1 - \lambda)/3 & 0\\0 & \alpha} \pmatrix{1&1&1\\1&-1&0}.
$$
A: The list need not be finite. Here is an example:
$$P:=\begin{pmatrix}\tfrac{1}{4}&\tfrac{1}{4}& \tfrac{1}{2} \\
 \tfrac{1}{2} & \tfrac{1}{4} & \tfrac{1}{4} \\
 \tfrac{1}{2} & \tfrac{1}{2} & 0\end{pmatrix}$$
It is not diagonalizable, and the coefficients of $P^n$ do not repeat (the denominator of $P_{11}$ grows as $4^n$).
