Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Is it that the values in each column add up to one? Also, how does this work?

share|improve this question

3 Answers 3

up vote 3 down vote accepted

The column sums of the transition matrix are not relevant to your problem.

Here is Theorem (3) from section 6.4 of Probability and Random Processes (3rd edition) by Grimmett and Stirzaker.

An irreducible chain has a stationary distribution $\pi$ if and only if all the states are non-null persistent; in this case, $\pi$ is the unique stationary distribution and is given by $\pi_i=\mu_i^{-1}$ for each $i\in S$, where $\mu_i$ is the mean recurrence time of $i$.

Their term "non-null persistent" is often called "positive recurrent" by other authors. This stationary distribution $\pi$ will also be the limiting distribution (i.e. $X_n\to \pi$ in distribution) provided that the chain is, in addition, aperiodic.


Added: On re-reading this, it looks like I've sidestepped the question but substituting one abstract probabilistic condition for another. Let me try to answer in the OP's language.

I will assume a finite state space.

What conditions on the transition matrix $P$ guarantee the existence of a limiting distribution? That is, $\pi=\lim_n {\cal L}(X_n)$ where $\cal L$ means "law" and the limit is "in distribution". Since we do not specify the initial state, we will need that every recurrent state is aperiodic.

In linear algebra terms, this is true if and only if every eigenvalue (except 1) of $P$ has modulus strictly less than 1.

These references may be helpful: Perron–Frobenius theorem, Markov Chains in discrete time.

share|improve this answer
    
Thanks, that was very helpful. –  Beatrice Mar 22 '11 at 21:22
    
+1. I wonder in your added part, how to prove that $\lim_{n \to \infty} P^n$ have identical rows, ie have a limiting distribution, when every eigenvalue (except 1) of P has modulus strictly less than 1? –  Tim May 1 '12 at 14:34
    
@Tim I only say that $\lim_n P^n$ exists, not that it has identical rows. The rows will be identical when the eigenvalue 1 has multiplicity one. –  Byron Schmuland May 2 '12 at 2:57

I've not done Markov processes before but if you look here it seems that

(i) the rows sum to 1 because otherwise the transition probabilities are not a probability distribution

(ii) to compute the stationary distribution you have to compute $\mathbb{\pi}$ as left eigenvector of the transition matrix $P$: $\mathbb{\pi} P = \mathbb{\pi}$

Does that make any sense? If it does then the answer to your question would be that the stationary distribution exists if $P$ has a left eigenvector.

share|improve this answer
1  
The transition probabilities are in the matrix rows, not the columns. If the columns sums are also 1, the transition matrix is called doubly stochastic. –  Byron Schmuland Mar 22 '11 at 13:43
    
@Byron: thanks Byron, I corrected it. –  Rudy the Reindeer Mar 22 '11 at 13:57

In short, every irreducible + aperiodic transition matrix (primitive matrix as Wikipedia) will have a stationary distribution vector. These Wikipedia links will help you to have a good perspective in Matrix algebra terms: https://en.wikipedia.org/wiki/Stochastic_matrix https://en.wikipedia.org/wiki/Perron%E2%80%93Frobenius_theorem

For a statistical interpretation of these results, I recommend this very short read: Finite Markov Chains and Algorithmic Applications - Olle Haggstrom. London Math Society. Student Texts 52. Chapter 4 and 5.

Best regards! Eric.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.