0
$\begingroup$

Let be a finite time homogeneous Markov chain with $P_{n \times n}$ transition matrix. As far as I know, $P$ has a unique stationary distribution if and only if it has a unique recurrent class (***).

So, let’s take: $$P=\begin{pmatrix} 3/4&0&1/4 \\ 1/3&1/2&1/6 \\ 1&0&0 \end{pmatrix}$$ State $2$ is transient, so the chain can’t have a unique recurrent class, right?

Now, if I calculate a stationary vector $\pi = (a, b, c)$ by $(a \ b \ c)P=(a \ b \ c)$, I find only one such vector $\pi = (34/5, 0, 1/5)$.

Isn't there a contradiction with statement (***) ?

I'm quite worried because there's obviously something I don't understand... but what the hell am I not understanding ?

$\endgroup$

1 Answer 1

0
$\begingroup$

This is an absorbing Markov chain, i.e. there exists a state $i$ such that $\mathbb P(X_n = i \mid X_0=1)=1$ for all $n>0$. For clarity of notation I'll swap columns $0$ and $2$ to write $$ P = \left( \begin{array}{ccc} \frac{1}{4} & 0 & \frac{3}{4} \\ \frac{1}{6} & \frac{1}{2} & \frac{1}{3} \\ 0 & 0 & 1 \\ \end{array} \right)=\begin{pmatrix} Q & R \\ \mathbf 0 & I\end{pmatrix} $$ where $Q$ is the substochastic matrix obtained from $P$ by considering transitions betweeen transient states, $R$ that obtained from $P$ by considering transitions from a transient state to an absorbing state, $\mathsf 0$ that obtained by considering transitions from absorbing states to transient states, and $I$ that obtained by considering transitions between absorbing states (which with reordering of states can be written as the identity matrix with dimension the number of absorbing states).

By definition, a stationary distribution $\pi$ for a Markov chain is a row vector $\pi$ satisfying $\sum_i=1$ with $\pi>0$ and $\pi P=\pi$. Iteratively this would imply that $\pi P^n=\pi$ for all positive integers $n$, but as state $2$ is absorbing and $P_{02}$ and $P_{12}$ are positive, for any distribution $\pi$ there exists some positive integer $N$ such that $\pi P^n=\begin{pmatrix} 0 & 0 & 1\end{pmatrix}$ for $n\geqslant N$, meaning there can be no stationary distribution for this Markov chain.

In general, stationary distributions for finite Markov chains exist if and only if the chain is irreducible, in which case the stationary distribution is unique if and only if the chain is aperiodic (this is a good exercise to prove).

Edit: The above answer is considering (positive) recurrent Markov chains. As shown in this answer, $\begin{pmatrix} 0 & 0 & 1\end{pmatrix}$ is a valid stationary distribution for $P$.

$\endgroup$
2
  • $\begingroup$ Thanks for the detailed answer. I thought that $stationary \ \pi \Longleftrightarrow \sum_i = 1, \pi P = \pi$, but you added the condition $\pi >0$. In this case, my calculated vector cannot be rated as "stationary", which resolves the question... Am I right? $\endgroup$
    – Andrew
    Jan 25 at 19:47
  • $\begingroup$ I was a bit imprecise in my answer. A Markov chain with transient states can have elements equal to zero in a stationary distribution (in particular, those corresponding to the transient states). $\endgroup$
    – Math1000
    Jan 28 at 4:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .