# Markov Chain probabilities

I'm having trouble with this problem from Resnick's Adventures in Stochastic Processes:

Consider a Markov Chain on states {0,1,2} with transition matrix $\left( \begin{array}{ccc} 0.3 & 0.3 & 0.4 \\ 0.2 & 0.7 & 0.1 \\ 0.2 & 0.3 & 0.5 \end{array} \right)$.

Compute $P[X_{16}=2|X_0=0]$ and $P[X_{12}=2,X_{16}=2|X_0=0]$. Try not to do this by hand.

I don't know how to set up this problem-I don't understand how to solve this probability for a large n. Do I need to use the Chapman Kolmogorov equation? Thanks for any help

• How would you solve the problem for small values of $n$? – Gareth Mar 9 '14 at 8:54

Questions about $X_{16}$ relate to the 16th power of the transition matrix. "Try not to do this by hand" may just mean, find some software that can compute this 16th power for you. On the other hand, it may mean, try to get an answer without computing the 16th power at all, instead assuming that by the time $n=16$, everything has converged to the steady state. If that's what it means, then what you have to do is construct a (left-)eigenvector for the eigenvalue 1, which is to say you have to subtract the identity matrix from the transition matrix and then find a probability vector in the (left-)nullspace of the resulting matrix.