Transition probability matrix of a Markov chain.

Question

I am not understanding how is the transition probability matrix of the following example constructed.

Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$; if it rained today but not yesterday, then it will rain tomorrow with probability $0.5$; if it rained yesterday but not today, then it will rain tomorrow with probability $0.4$; if it has not rained in the past two days, then it will rain tomorrow with probability $0.2$.

Let

state $0$ if it rained both today and yesterday,

state $1$ if it rained today but not yesterday,

state $2$ if it rained yesterday but not today,

state $3$ if it did not rain either yesterday or today.

The preceding would then represent a four-state Markov chain having a transition probability matrix

$$P= \begin{bmatrix} 0.7 & 0 & 0.3 & 0 \\ 0.5 & 0 & 0.5 & 0 \\ 0 & 0.4 & 0 & 0.6 \\ 0 & 0.2 & 0 & 0.8 \\ \end{bmatrix}. $$

Why is $P_{10}=0.5$ ? As the 2nd row corresponding to state $1$ represents it rained today but not yesterday, can't i assign the $0.5$ in $P_{11}$ or in $P_{13}$ ?

$\bullet$ Second portion of the example is : Given that it rained on Monday and Tuesday, what is the probability that it will rain on Thursday?

For solving this we have to compute two-step transition matrix. But if I were asked to find the probability that it will rain on Friday, would I have to compute three-step transition matrix ? Will I count the step from Tuesday?

And the book says that rain on Thursday is equivalent to the process being in either state $0$ or state $1$. Why it is not other states rather than state $0$ or state $1$?

Answer 1

First part:

Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = \text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = \text{(rain, rain)}$. Then we have the sequence $(a,b,c) = \text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.

Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = \text{(no rain, rain)}$ and $(b,c) = \text{(no rain, rain)}$, which can't happen.

---

Second part:

Notice that we start from state $0$, thus $\pi(0) = \begin{bmatrix} 1& 0 & 0 & 0\end{bmatrix}$ and we are going to evaluate the probability: $$\pi(0)\cdot P^2 = \begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18\end{bmatrix}. $$ Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).

---

Third part:

From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold: $$(a,b,c,d) = \text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.

1: $(a,b,c,d) = \text{(rain, rain, rain, rain)}$

2: $(a,b,c,d) = \text{(rain, rain, no rain, rain)}$

Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.

Speaking with term of states the first $4-tuple$ corresponds to the path $0\to 0\to 0$, thus we have $p_{00}\cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0\to 2\to 1$, thus $p_{02}\cdot p_{21} = 0.3 \cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.

Answer 2

1) Refer to the probability tree diagram:

$\hspace{3cm}$

Let (yesterday, today): $S_0$: (R, R); $S_1$: (N, R); $S_2$: (R, N); $S_3$: (N, N).

Then (yesterday, today, tomorrow): $$\begin{align} (R,R,R) &\Rightarrow S_0 \stackrel{0.7}{\to} S_0 \Rightarrow P_{00}\\ (R,R,N) &\Rightarrow S_0\stackrel{0.3}{\to} S_2 \Rightarrow P_{02}\\ (R,N,R) &\Rightarrow S_2\stackrel{0.4}{\to} S_1 \Rightarrow P_{21}\\ (R,N,N) &\Rightarrow S_2\stackrel{0.6}{\to} S_3 \Rightarrow P_{23}\\ (N,R,R) &\Rightarrow S_1\stackrel{0.5}{\to} S_0 \Rightarrow P_{10}\\ (N,R,N) &\Rightarrow S_1\stackrel{0.5}{\to} S_2 \Rightarrow P_{12}\\ (N,N,R) &\Rightarrow S_3\stackrel{0.2}{\to} S_1 \Rightarrow P_{31}\\ (N,N,N) &\Rightarrow S_3\stackrel{0.8}{\to} S_3 \Rightarrow P_{33}\\ \end{align}\\ P_{01}=P_{03}=P_{11}=P_{13}=P_{30}=P_{32}=P_{40}=P_{42}=0.$$ Hence: $$P= \begin{bmatrix} 0.7 & 0 & 0.3 & 0 \\ 0.5 & 0 & 0.5 & 0 \\ 0 & 0.4 & 0 & 0.6 \\ 0 & 0.2 & 0 & 0.8 \\ \end{bmatrix}.$$ 2) Probability of raining on Thurday given that it rained on Monday and Tuesday: $$\mathbb P(\underbrace{RR}_{S_0}\underbrace{RR}_{S_0})+\mathbb P(\underbrace{RR}_{S_0}\underbrace{NR}_{S_1})=0.7\cdot 0.7+0.3\cdot 0.4=0.61.$$ which is: $$\begin{bmatrix} \underbrace{1}_{S_0} & \underbrace{0}_{S_1} & \underbrace{0}_{S_2} & \underbrace{0}_{S_3} \end{bmatrix} \begin{bmatrix} 0.7 & 0 & 0.3 & 0 \\ 0.5 & 0 & 0.5 & 0 \\ 0 & 0.4 & 0 & 0.6 \\ 0 & 0.2 & 0 & 0.8 \\ \end{bmatrix}^2= \begin{bmatrix} \underbrace{0.49}_{S_0} & \underbrace{0.12}_{S_1} & \underbrace{0.21}_{S_2} & \underbrace{0.18}_{S_3}\end{bmatrix} \Rightarrow\\ \mathbb P(S_0)+\mathbb P(S_1)=0.49+0.12=0.61.$$ 3) Probability of raining on Friday given that it rained on Monday and Tuesday. Can you figure this out both ways: probability tree diagram and markov chain formula? The initial state matrix is $(1,0,0,0)$, which must be multiplied by the cube of the transition matrix and the probabilities of $S_0$ and $S_1$ must be added.