Why is the stationary distribution of a Markov Chain one over the mean sojourn time? I am reading the book "Markov Chains and Mixing Times" by Levin et al. and i have a problem with proposition 1.14, page 12.
There a stationary distribution is constructed by defining for fixed initial state z
$$\tilde \pi(y):=\mathbf E_y(\text{number of visits to y before returning to z})=\sum_{t=0}\mathbf P(X_t=y,\tau_z^+>t)$$
$\tau_z^+$ is the first return time to z.
The proof is all well, but in the end a stationary distribution $\pi$ is constructedby normalizing such that
$$\pi(x)=\frac{\tilde \pi(x)}{\mathbf E_z(\tau_z^+)}$$
Then, without further comment it is stated that this means in particular that
$$\pi(x)=\frac{1}{\mathbf E_x(\tau_x^+)}$$
How do i see this?
 A: The statement is true, but I think the way it is presented is a mistake by the authors. What is obvious is that
$$\pi(z) = \frac{1}{\mathbb{E}_z (\tau_z^+)}.$$
Indeed, starting from $z$, the Markov chain visits $z$ exactly once (at time $t=0$)  before returning to $z$, so $\tilde{\pi} (z) = 1$.
However, the stationary measure we got depends a priori on the state $z$ chosen to construct it. If we use another state $z'$, then we get another stationary measure $\pi'$ such that $\pi'(z') = \frac{1}{\mathbb{E}_{z'} (\tau_{z'}^+)}$, but there is no guarantee that $\pi'(z') = \pi(z')$! This is actually false for certain examples of Markov chains (take e.g. a Markov chain on two states with the identity as the transition matrix).
What saves us is that, if the Markov chain is irreducible, there is indeed a unique stationary probability measure, which implies that $\pi = \pi'$. Anyway, at this point, the proof must somehow use that fact that the Markov chain is irreducible.
I think that this is a mistake by the authors since they do not explicitely mention the irreducibility at this point of the proof, and the uniqueness is only proved in the following section.
