I'm new to probability models.

Taken from Ross' Introduction to Probability Models 11th ed:

"Example 4.4 (Transforming a Process into a Markov Chain)

Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability 0.7; if it rained today but not yesterday, then it will rain tomorrow with probability 0.5; if it rained yesterday but not today, then it will rain tomorrow with probability 0.4; if it has not rained in the past two days, then it will rain tomorrow with probability 0.2. If we let the state at time n depend only on whether or not it is raining at time n, then the preceding model is not a Markov chain (why not?)."

I know that a Markov chain as defined in the book must satisfy the Markovian property, i.e. given the present, the future is independent of the past, and that there is a fixed prob. of going from state i to state j, but I can't seem to connect this to the example. Any help?

  • 1
    $\begingroup$ It says "If we let the state at time $n$ depend ONLY on whether or not it is raining at time $n$ ...''. Given this state, the probability of rain at time $n+1$ is not independent of the past, because it depends also on whether it has rained at time $n-1$. $\endgroup$
    – Gerhard S.
    Mar 10, 2019 at 15:57
  • 1
    $\begingroup$ What is the exact wording of the Markovian property in your book? Because this question is possibly about nitpicking on a word-by-word level, and we can't help you do that without knowing which words to nitpick about. $\endgroup$
    – Arthur
    Mar 10, 2019 at 16:06
  • $\begingroup$ @Arthur Let $\{X_n, n = 0,1,2,\dots\}$ be a stochastic process that takes on a finite or countable number of possible values. If $X_n = i$, then the process is said to be in state $i$ at time $n$. We suppose that whenever the process is in state $i$, there is a fixed probability $P_{ij}$ that it will next be in state $j$. That is, we suppose that $P\{X_{n+1} = j | X_n = i, X_{n-1} = i_{n-i}, \dots, X_1 = i_1, X_0 = i_0\} = P_{ij}$ for all states $i_0,i_1,\dots,i_{n-1},i,j$ and all $n \geq 0$. Such a stochastic process is known as a Markov chain. $\endgroup$
    – goblinb
    Mar 10, 2019 at 17:33

1 Answer 1


My (possible) interpretation is that, while it is perfectly possible to express the chain by a $4 \times 4$ matrix relating $4$-D vectors representing the status of two consecutive days $(rain \& rain, rain \& not-rain, \cdots)$, you cannot represent it by $2$-D vectors relevant only to the state of a single day.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.