A Markov chain $X_0$, $X_1$, $X_2$, ... has the transition probability matrix

$$ P = \left[ \matrix { 0.3&0.2&0.5 \\ 0.5&0.1&0.4 \\ 0&0&1 } \right] $$

and is known to start in state $X_0$ = 0. Eventually, the process will end up in state 2. What is the probability that when the process moves into state 2, it does so from state 1?


Let $\tau = \min\{n: X_{n+1}= 2\}$ and $p_{i,j} = P(X_{\tau}=i|X_{0}=j)$. Observe that:

$$p_{0,0} = P(X_{\tau}=0|X_{0}=0) = \sum_{k=0}^{2}{P(X_{\tau}=0,X_{1}=k|X_{0}=0)} =$$


Do you know $P(X_{1}=k|X_{0}=0)$? Can you simplify $P(X_{\tau}=0|X_{1}=k,X_{0}=0)$ in terms of $p_{0,0}$ and $p_{0,1}$? Do the same steps for $p_{0,1}$ and solve the system of equations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.