Proving a Trick to More Quickly Calculate N-Step Transition Probabilities So, I have been working on a homework problem all day that asks me to prove that: $P^n= \Pi +Q^n$
where P is the transition matrix of a finite-state regular Markov Chain, $\Pi$ is a matrix whose rows are the stationary distribution of P, and $Q = P - \Pi$.
The fundamental relation that I have been attempting to show is that $(P-\Pi)^n=P^n-\Pi$. I'm not sure if there is a relevant matrix product property here that would allow me to establish this relation but I can't figure out how to proceed. Also, for those who have math backgrounds but not necessarily statistics backgrounds, the stationary distribution is just a left eigenvector with eigenvalue 1, and P is a stochastic matrix (rows summing to 1).
Originally, I was attempting to rewrite the matrix product relation in sigma notation and try to achieve some sort of simplification. I didn't find anything by doing that, so here is my most current attempt:
$P^n= \Pi +Q^n$
Substituting in for Q to get to the relation I referred to earlier:
$P^n=\Pi+(P-\Pi)^n$
$P^n-\Pi=(P-\Pi)^n$
Then, I use the property that the eigenvalue of $\Pi$ is 1 to rewrite P as $\Pi*P$.
$(\Pi*P-\Pi)^n=P^n-\Pi$
$\Pi^n*(P-1)^n=P^n-\Pi$
From here, I am unsure how to proceed but believe that this is a natural way to use the eigenvalue to alter the relation.
 A: I understand matrix $\Pi$ to be:
$$
\Pi =   \begin{bmatrix}
        \pi_0 & \pi_1 & \cdots & \pi_m \\
        \pi_0 & \pi_1 & \cdots & \pi_m \\
        \cdots &  & & \\
        \pi_0 & \pi_1 & \cdots & \pi_m \\
        \end{bmatrix}
$$
The property of $\Pi$ that: $\;\Pi P = P\Pi = \Pi\;\Pi  = \Pi\;$ is easy to prove. We have 
$$Q^n = (P-\Pi)^n$$
and if we expand that expression, using the above property we get much simplification (any term containing at least one factor of $\Pi$ reduces to $\pm\Pi$) and our expansion finishes up with the following terms:
\begin{align}
2^{n-1} & \;\text{ instances of } -\Pi \\
2^{n-1}-1 & \;\text{ instances of } \Pi \\
1 & \;\text{ instance of } P^n.
\end{align}
So we get 
$$\Pi + Q^n = \Pi + (-\Pi + P^n) = P^n.$$

Proof of above property:


*

*$\Pi P=\Pi$: this is part of the definition of a stationary distribution.

*$P\Pi = \Pi$: the $(i,j)^{th}$ entry of $P\Pi$ by matrix multiplication is:
$$\sum_{k} p_{ik}\cdot\pi_j = \pi_j\sum_{k} p_{ik} = \pi_j = (i,j)^{th} \text{ entry of } \Pi.$$


*$\Pi\;\Pi = \Pi$: the $(i,j)^{th}$ entry of $\Pi\;\Pi$ by matrix multiplication is:


$$\sum_{k} \pi_{k}\cdot\pi_j = \pi_j\sum_{k} \pi_{k} = \pi_j = (i,j)^{th} \text{ entry of } \Pi.$$
