I am slightly confused about the solution to Exercise 11 in Chapter 4 of Introduction to Probability Models, by Sheldon M. Ross.


On any given day Gary is either cheerful (C), so-so (S), or glum (G). If he is cheerful today, then he will be C, S, or G tomorrow with respective probabilities 0.5, 0.4, 0.1. If he is feeling so-so today, then he will be C, S, or G tomorrow with probabilities 0.3, 0.4, 0.3. If he is glum today, then he will be C, S, or G tomorrow with probabilities 0.2, 0.3, 0.5. Letting $X_n$ denote Gary’s mood on the nth day, then $\{X_n, n \geq 0$} is a three-state Markov chain (state 0 = C, state 1 = S, state 2 = G) with transition probability matrix $$P = \begin{pmatrix} 0.5 & 0.4 & 0.1 \\ 0.3 & 0.4 & 0.3 \\ 0.2 & 0.3 & .5 \end{pmatrix}. $$

The question:

In Example 4.3, Gary was in a glum mood four days ago. Given that he hasn’t felt cheerful in a week, what is the probability he is feeling glum today?

According to the solution manual, the answer is $\frac{P_{2,2}^4}{1-P_{2,0}^4}$, with $$P = \begin{pmatrix} 1 & 0 & 0 \\ 0.3 & 0.4 & 0.3 \\ 0.2 & 0.3 & .5 \end{pmatrix}.$$

However, my initial approach had been to form a new, reduced transition probability matrix $$P = \begin{pmatrix} \frac{4}{7} & \frac{3}{7} \\ \frac{3}{8} & \frac{5}{8} \end{pmatrix},$$ where I just completely neglect the possibility of going into state 0, and simply report $P_{1,1}^4$ as the answer. Why is this approach wrong? To be clear, I fully understand the logic behind the correct answer, and I know something is not right with the way I tried to solve the problem, but I would appreciate a clear explanation as to why it doesn't work.


1 Answer 1


I think the following example can maybe illustrate what is happening.

John works at noob corporation. His boss, in typical fashion is annoying.

There are three "moods" in which John can be, they are Fired,angry and happy with transitions given by this matrix:

$\begin{pmatrix} 1 & 0 & 0 \\ 0.5 & 0.4 & 0.1 \\ 0 & 0.2 & 0.8 \\ \end{pmatrix}$

So, if he's fired he stays fired, and it is only possible for him to be fired if he is angry the previous day.

Let's assume we want to calculate the probability he is happy after $N$ days given he hasn't been fired and he is happy at the start.

If we use your approach then we get the probability goes to $\frac{1}{2}$ for $N$ large.

However for large $N$ the real probability is larger.

This is because the sequences with a lot of Angry states are actually less likely conditioned to him never getting fired.

In your approach the probability that John starts out angry and ends angry is the same as the probability that John starts happy and ends happy.

However staying angry for $N$ consecutive days is a lot less likely than staying happy for $N$ consecutive days for example.

  • $\begingroup$ Maybe this computation can drive the point home a bit more. Take $N=30$ and compute the probability that we are happy after $20$ days given we never get fired and we started happy. With the incorrect method we get it's pretty darn close to $\frac{1}{2}$. With the correct method we get approximately $0.68$ $\endgroup$
    – Asinomás
    May 20, 2021 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.