0
$\begingroup$

A Markov Chain $(X_n)_n$ has the following transition matrix:
$$P = \begin{bmatrix} 0.1 & 0.3 & 0.6\\ 0 & 0.4 & 0.6\\ 0.3&0.2&0.5 \end{bmatrix}$$ with initial distribution $\alpha = (0.2, 0.3, 0.5)$.

Find the following:
(a) $P(X_7 = 3|X_6 = 2)$
(b) $P(X_9 = 2|X_1 = 2, X5 = 1, X_7 = 3)$
(c) $P(X_0 = 3|X_1 = 1)$

What I understand according to the Example-$7.16$ of book by Symour Lipchuz (Page-$132$) that, $P^n$ is used to find the probabilities of change of states in exactly $n$ steps, and $p^{(m)} \cdot P^n$ is used to find the probability distribution of various states after $m$ steps.

According to my understanding, there are no uses of Initial Distribution in these three computations. Because, multiplication of a $1X3$ and $3X3$ matrices will give a $1X3$ matrix. In that case, the transition matrix won't make sense for three states.

So, my attempted solution is the following:


(a) This question actually asks:

What is the probability that the system changes from state-$2$ to state-$3$ in exactly $1$ step?

So, the answer would be $p(2, 3) = 0.6$

(b) This question actually asks:

What is the probability that the system changes from state-$3$ to state-$2$ in exactly $2$ steps?

$$P^2 = P \cdot P = \begin{bmatrix} 0.1 & 0.3 & 0.6\\ 0 & 0.4 & 0.6\\ 0.3&0.2&0.5 \end{bmatrix} \cdot \begin{bmatrix} 0.1 & 0.3 & 0.6\\ 0 & 0.4 & 0.6\\ 0.3&0.2&0.5 \end{bmatrix} = \begin{bmatrix} 0.19&0.27&0.54\\ 0.18&0.28&0.54\\ 0.18&0.27&0.55 \end{bmatrix} $$

So, the answer would be: $p(3, 2) = 0.27$

(c) I think, this question asks:

What is the probability that the system changes from state-$1$ to state-$3$ in exactly $9$ steps?

Is it? If yes, then it would be $p(1,3)$ from $P^9$.

Am I correct?


Edit:

$P(X_0 = 3|X_1 = 1) = \frac{P(X_0 = 3, X_1 = 1)}{P(X_1 = 1)}$
$\Rightarrow P(X_0 = 3|X_1 = 1) = \frac{P(X_1 = 1, X_0 = 3)}{P(X_1 = 1)}$
$\Rightarrow P(X_0 = 3|X_1 = 1) = \frac{P(X_1 = 1| X_0 = 3)\cdot P(X_0=3)}{P(X_1 = 1)}$

Now,

  • from $P$, we have $P(X_1=1|X_0=3) = 0.30$
  • from $\alpha$, we have $P(X_0=3) = 0.50$, and
  • from $\alpha P$, we have $P(X_1=1) = 0.17$

So,

$P(X_0 = 3|X_1 = 1) = \frac{0.30 \cdot 0.50}{0.17} \approx 0.88$.

Is this a correct calculation?

$\endgroup$
  • $\begingroup$ At the top, you have for (c) $P(X_0=3 \mid X_1=1)$, but later you talk about 9 steps. Presumably this means that you’re thinking about $X_9$ (in which case it’s really eight steps) or $X_{10}$. Which is it? $\endgroup$ – amd Mar 10 at 23:17
  • $\begingroup$ No, they’re not ”changing in reverse order.” The process only goes in one direction. It’s asking you about the probability that the system started in state 3 given that it’s in state 1 after a single step. Why then are you asking about nine steps of the process at the end of your question? $\endgroup$ – amd Mar 10 at 23:20
  • $\begingroup$ $X_0$ is the state at time $t=0$; $X_9$ is the state at time $t=9$. You can’t go back to $0$ from $9$. $\endgroup$ – amd Mar 10 at 23:31
  • $\begingroup$ It means exactly what I wrote in a previous comment. You’re being asked about the probability of a specific history of the system. $\endgroup$ – amd Mar 10 at 23:39
1
$\begingroup$

You have part (a) and part (b) down. For part (c), we need that initial distribution.

What is the probability that the system is in state 1 after one step?
What is the probability that the system starts in state 3 and is then in state 1 after 1 step?
Can you use these to find the conditional probability this part asks for?

And no, there's no cyclic behavior here. The sequence of $X_n$ keeps going for all positive integers $n$, and it's not periodic.

[In response to the added material in the question]
Yes, that's a correct calculation. You have now fully solved the problem.

$\endgroup$
  • $\begingroup$ Considering that your claimed answers imply the probability of starting in state 3 and then going to state 1 is less than the probability of going to state 1 regardless of where we started ... no, you've got that part wrong. $\endgroup$ – jmerry Mar 10 at 23:36
  • $\begingroup$ I'm not sure what you were trying to say there. Now, what I was trying to say with the bit that had the word "regardless"? The probability of starting in state 3 and then going to state 1 after one step must be $\le$ the probability of being in state 1 after one step. $\endgroup$ – jmerry Mar 11 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.