# Forming the transition matrix for Markov chain, given a word description of transition probabilities

I have just started learning about Markov chain and have a trouble determining appropriate transition matrix:

Suppose that whether or not it rains today depends on previous weather conditions through the last three days. Show how this system may be analyzed by using a Markov chain by determining how many states are needed and describing the general form of the P matrix. Also suppose that if it has rained for the past three days, then it well rain today with probability .8; if it did not rain for any of the past three days, then it will rain today with probability.2; and if any other case, the weather today will, with probability .6., be the same as yesterday. Determine the matrix P for this Markov chain.

This question has a very detailed explanation but I still don't get it.

Can someone please explain me at least one row? Also how are you suppose to read notations, does YYYY= P(00) means that it will rain today given that it has rained for the past three days? For instance, P(00) = Y YYY = 0.8 because it has rained for the past three days (YYY) then it will rain today.

P(10) = Y NYY = 0. Why 0? Why it isnt 0.6? but P(30) = Y YYN =0.6 I have spent so much time looking for explanations but every website just gives transition matrix without explaining me obtained values.

• Minor note: a Markov chain is named after Andrey Markov, also spelled Andrey Markoff
– Tom
Commented Dec 1, 2013 at 2:53

Suppose it is Friday (Fr), and you are asked to determine the probability that it will rain tomorrow on Saturday (Sa). You will need to know whether it rained yesterday, Thursday (Th), and two days ago on Wednesday (We). Given the information in the problem, you can make the following table:

$$\begin{array} {c|c} \text{Rained on} & \text{Will Rain on} \\ \begin{array} {ccc} \text{We} & \text{Th} & \text{Fr} \\ \hline Y & Y & Y \\ Y & Y & N \\ Y & N & Y \\ Y & N & N \\ N & Y & Y \\ N & Y & N \\ N & N & Y \\ N & N & N \\ \end{array}& \begin{array} {c} \text{Sa} \\ \hline 0.8 \\ 0.4 \\ 0.6 \\ 0.4 \\ 0.6 \\ 0.4 \\ 0.6 \\ 0.2 \\ \end{array} \end{array}$$

Now this is a transition table , but the output states are not the same as the input states. The input states are the rain of 3 days; the input states are the rain of 1 day. So to make a transition matrix, we need the output states to be the same as the input states: 3 days to 3 days, {We, Th, Fr} to {Th, Fr, Sa}.

$$\begin{array} {c|c} \text{Rained on} & \text{Will Rain on Th Fr Sa } \\ \begin{array} {ccc} \text{We} & \text{Th} & \text{Fr} \\ \hline Y & Y & Y \\ Y & Y & N \\ Y & N & Y \\ Y & N & N \\ N & Y & Y \\ N & Y & N \\ N & N & Y \\ N & N & N \\ \end{array}& \begin{array} {c} \text{YYY} & \text{YYN} & \text{YNY} & \text{YNN} & \text{NYY} & \text{NYN} & \text{NNY} & \text{NNN} \\ \hline 0.8 & 0.2 \\ & & 0.6 & 0.4 \\ & & & & 0.6 & 0.4 \\ & & & & & & 0.6 & 0.4 \\ 0.6 & 0.4 \\ & & 0.6 & 0.4 \\ & & & & 0.6 & 0.4 \\ & & & & & & 0.2 & 0.8 \\ \end{array} \end{array}$$

Each blank entry is an impossible transition. For example, it's impossible to transition from a Friday where it rained 3 days in a row to a Saturday where it was clear 3 days in a row (the top right corner). So your final transition matrix is:

$$\begin{bmatrix} 0.8 & 0.2 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0.4 & 0.6 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.6 & 0.4 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.4 & 0.6 \\ 0.6 & 0.4 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0.4 & 0.6 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.6 & 0.4 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.2 & 0.8 \\ \end{bmatrix}$$

The "notation" (if it can even be called that) that your guide is using is replacing the yes/no sequence with a number (So YYY is 0, NNN is 7, it is quite confusing). $$P_{0,0}$$ is the probability that it rained for the past 3 days and tomorrow will also be a day where it has rained for 3 days.

• Hi @DanielV I know that you wrote this post 8 years ago, but it was today extremely usefull to me. I had an additional question for which I made a new post. I you see fit, could you please give me feedback on my approach? math.stackexchange.com/questions/4562371/…
– Tim
Commented Oct 26, 2022 at 20:49

Let's agree first that a reasonable state space is $$S = \{YYY, YYN, YNY, NYY, YNN, NYN, NNY, NNN\}$$ where, for example, $YYY$ means it rained today and the previous two days, whereas $YYN$ means that it didn't rain today, but did rain the previous two days, etc. To be clear the transitions will be to move to the weather tomorrow, today and the day before. So, for example, $YYY \to YYN$ means the transition from the state where today and the previous two days rained into the state where it doesn't rain tomorrow (the "next step's today") and did rain the previous two days. Now $P(YYY \to YYN) = .2$ whereas $P(YYY \to YYY) = .8$ by your story. However, state changes as $P(\ast \ast Y \to \ast N \ast) = 0$ since there is no way that if it rained today, then you transition to where it doesn't rain today (I used the symbol $\ast$ means any weather).. does this make sense? If so, I imagine using $S$ as I have listed, you can create the transition matrix.