Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm reading through my textbook, Introduction to Stochastic Processes (Lawler), before the semester begins in hopes of getting ahead, and I've run into something I just plain cannot figure out: How to compute the invariant probability vector for a transition matrix. I was hoping that one (or many) of you would be able to walk me through how you would do this for just a simple matrix:

$$\begin{bmatrix} .4&.2&.4 \\\\ .6&0&.4 \\\\ .2&.5&.3 \end{bmatrix}$$

I know that you can compute it by raising the matrix to a large power, but this practice problem says to "compute the invariant probability vector as a left eigenvector." How would one go about doing this?

Thanks for your help!

share|improve this question
    
Invariant: matrix operation on $\mu$ still gives $\mu$. –  FrenzY DT. Aug 12 '12 at 23:53

3 Answers 3

up vote 6 down vote accepted

If the transition matrix is $A$ and the probability vector is $\mu$, "invariant" means that $\mu A = \mu$. Another way of saying this is that $\mu$ is a left eigenvector of $A$ with eigenvalue 1.

$\mu A = \mu$ is really just a system of linear equations. If we write $\mu = [\mu_1, \mu_2, \mu_3]$ then we have $$[\mu_1, \mu_2, \mu_3] \begin{bmatrix} .4&.2&.4 \\\\ .6&0&.4 \\\\ .2&.5&.3 \end{bmatrix}= [\mu_1, \mu_2, \mu_3]$$ or in other words $$\begin{align*} .4 \mu_1 + .6 \mu_2 + .2 \mu_3 &= \mu_1 \\ .2 \mu_1 + 0 \mu_2 + .5 \mu_3 &= \mu_2 \\ .4 \mu_1 + .4 \mu_2 + .3 \mu_3 &= \mu_3. \end{align*} $$ Since $\mu$ is to be a probability vector we also have to have $$\mu_1 + \mu_2 + \mu_3 = 1.$$ So you have a system of 4 linear equations in 3 unknowns. Now you just have to solve this system.

share|improve this answer
    
Ahh this makes so much more sense now. Thank you! –  radcliffejh Aug 12 '12 at 23:43
1  
Note that the first three equations are linearly dependent since the sum of each row of the matrix is $1$, so you can take any two of those three plus the fourth equation, and solve $3$ equations in $3$ unknowns. –  Robert Israel Aug 13 '12 at 0:11

It means, find a vector $v$ with the following properties: all entries between 0 and 1, entries add up to 1, and $vA=v$ (where $A$ is your transition matrix).

If you know how to find (left) eigenvectors, you just find one for the eigenvalue 1, and then normalize it so the sum of the entries is 1.

share|improve this answer
    
The first part of your response makes sense, but before reading this book I'd never heard the term left eigenvector. I took Linear Algebra a year ago so I can find the eigenvalue, but I couldn't tell you how to find the left eigenvector. –  radcliffejh Aug 12 '12 at 23:47
1  
@JackRadcliffe: You probably know how to find right eigenvectors: a column vector $v$ such that $Av = \lambda v$ (here $\lambda = 1$). A left eigenvector is a row vector such that $vA = \lambda v$. And if you know how to find right eigenvectors (for instance, with your favorite software), then just note that the left eigenvectors of $A$ are just (the transposes of) the right eigenvectors of the transpose of $A$. –  Nate Eldredge Aug 13 '12 at 0:58
    
Yep, that's exactly what I know how to do. You've been very helpful; thanks again! –  radcliffejh Aug 13 '12 at 13:29

If T is the transistion probability matrix the stationary (or invariant) distribution satisfies the matrix equation

P(X)= T P(X)

share|improve this answer
    
I think you mean $P(X) T$? –  Nate Eldredge Aug 12 '12 at 23:41
1  
No, it's more than that. The left eigenvectors of $T$ are different from its right eigenvectors. (They are not just transposes of each other; compute them if you like. In particular, the right eigenvector of eigenvalue 1 isn't positive in this case.) The left eigenvectors of eigenvalue 1 correspond to invariant measures, and the right eigenvectors correspond to harmonic functions for the chain. It's essential to keep them straight. –  Nate Eldredge Aug 13 '12 at 0:55
1  
We're not understanding one another. I'm talking about a row vector $v$ (1x3) satisfying $vT = v$. These are different from the column vectors $w$ (3x1) which satisfy $Tw = w$. I really mean different; they have numerically different entries. And the former are what is needed to solve this problem, while the latter are what you are talking about. –  Nate Eldredge Aug 13 '12 at 4:15
1  
The standard presentation is that the entry in row $i$ and column $j$ is the probability of transitioning from $i$ to $j$. This is consistent with the matrix in the question; note that its rows sum to 1 while its columns do not. (Indeed, because of this the right eigenvectors will all be of the form $(c,c,c)^T$, which doesn't contain much useful information.) –  Nate Eldredge Aug 13 '12 at 13:17
1  
My experience is that mathematicians generally set up their stochastic processes with columns summing to 1 and $Av=v$ (like Michael), whereas statistics/economics people have rows summing to 1 and $vA=v$ (like OP). Causes some confusion when the two try to talk to each other. –  Gerry Myerson Aug 13 '12 at 22:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.