1
$\begingroup$

I'm having trouble with this problem from Resnick's Adventures in Stochastic Processes:

Consider a Markov Chain on states {0,1,2} with transition matrix $ \left( \begin{array}{ccc} 0.3 & 0.3 & 0.4 \\ 0.2 & 0.7 & 0.1 \\ 0.2 & 0.3 & 0.5 \end{array} \right)$.

Compute $P[X_{16}=2|X_0=0] $ and $P[X_{12}=2,X_{16}=2|X_0=0]$. Try not to do this by hand.

I don't know how to set up this problem-I don't understand how to solve this probability for a large n. Do I need to use the Chapman Kolmogorov equation? Thanks for any help

$\endgroup$
1
  • 1
    $\begingroup$ How would you solve the problem for small values of $n$? $\endgroup$
    – Gareth
    Commented Mar 9, 2014 at 8:54

1 Answer 1

Reset to default
1
$\begingroup$

Questions about $X_{16}$ relate to the 16th power of the transition matrix. "Try not to do this by hand" may just mean, find some software that can compute this 16th power for you. On the other hand, it may mean, try to get an answer without computing the 16th power at all, instead assuming that by the time $n=16$, everything has converged to the steady state. If that's what it means, then what you have to do is construct a (left-)eigenvector for the eigenvalue 1, which is to say you have to subtract the identity matrix from the transition matrix and then find a probability vector in the (left-)nullspace of the resulting matrix.

$\endgroup$
1
  • $\begingroup$ I'd add an additional possibility that in some situations the matrix may be diagonalizable (en.wikipedia.org/wiki/Diagonalizable_matrix) allowing matrix powers to be easily computed. $\endgroup$
    – Gareth
    Commented Mar 9, 2014 at 8:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .