0
$\begingroup$

A Markov chain has the transition probability matrix as follows. $$To$$ $$ From \begin{matrix} STATES& 0 & 1 & 2 \\ 0 & 0.6 & 0.3 & 0.1 \\ 1 & 0.3 & 0.3 & 0.4\\ 2 & 0.4 & 0.1 & 0.5\\ \end{matrix} $$ Assume that the initial value $X_0$ has the distribution: $P [X_0 = 0] = 0.3 , P [X_0 = 1] = 0.4 $, and$ P[X_0 = 2] = 0.3$

Find $P[X_0 = 0, X_1 = 2, X_2 = 1]$

I am just starting to learn how this stuff works. From my understanding I am finding $P[0.6, 0.4, 0.1]$ right? I need some help understanding how to move around the map.

$\endgroup$
1
$\begingroup$

The transition probability matrix tells you the probability of $X_n$ to be at state $k$ given that the previous time ($n-1$) you where at state $j$. So the probability you want is: $$P(X_0=0,X_1=2,X_2=1)=0.3\times 0.1\times 0.1$$

Note that $0.3$ is the probability that comes from the initial distribution.

The way of working with the transition matrix is: look at the transition matrix and see if you are in state $1$ for example go to the line that is state $1$ (in this matrix is the second row) and then if you want to go for example to state $0$ then go to the column of $0$ (in this matrix is the first one).

$\endgroup$
4
  • $\begingroup$ So it's saying that the prob. of being in state 0,0 is 0.3, prob. of being in state 0,1 is 0.4 and prob. of state 0,2 is 0.3? $\endgroup$ Feb 24 '14 at 20:05
  • $\begingroup$ what do you mean by state 0,0? $\endgroup$
    – Jimmy R.
    Feb 24 '14 at 20:06
  • $\begingroup$ like row 0 column 0 $\endgroup$ Feb 24 '14 at 20:06
  • 1
    $\begingroup$ There is no state $(0,0)$. At time $0$ you want to be at state $0$ so from the initia distribution find the appropriate probability. Now at time $1$ you want to go at state $2$ so go to transition matrix. But that at time $0$ you where at state $0$, this defines the row of the matrix that should look (row 1) and the fact that you want to go to state $2$ defines the column of the matrix (3 column). So you see that the probability is 0.1 $\endgroup$
    – Jimmy R.
    Feb 24 '14 at 20:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.