need help with a Markov chain for weather conditions

This is my second time posting the question as I failed to do so the first time, because I did not know the proper way. My apologies.

Question:

Suppose that whether or not it rains today depends on weather conditions through the previous teo days. If it has rained for the past two days, then it will rain today with probability 0.7. If it did not rain for any of the past two days, then it will rain today with probability 0.4. In any other case the weather today will, with probability 0.5, be the same as the weather yesterday. Describe the weather condition over time with a Markov chain.

Suppose that it rained today and yesterday. Calculate the expected number of days until it rains three consecutive days for the first time in the future.

I have found 4 different states that I named RR(0), RN(2), NR(1), and NN(3). R stands for when it rains and N is for when it does not.

As the question asks, I have tried finding the possible ways of three consecutive days being rainy. At time n, we are given it was rainy today and yesterday, meaning we are in State 0.

1-) First possibility is when it rains tomorrow, which gives us RRR (we got the three consecutive days)

2-) Second possibility is when we go from 0 to 2, from 2 to 1, from 1 to 0, and stay in 0 one day. That follows as : RRNRRR (In 4 days we can get rain for 3 consecutive days)

3-) Third is when we go from 0 to 2, from 2 to 3, from 3 to 1, from 1 to 0, and stay in 0 one day. That follows as: RRNNRRR.

To conclude what I have in mind about the question, wherever we go, when we get to State 0, we need to stay there one more day to get three consecutive rainy days. That means the minimum # of days to get rain for three consecutive days is one day. However, after this point, I am not able to proceed with the question.

Any help would be appreciated, thank you!!

Edit: I think the maximum # of days is just a random number, which leads me to the expectation of the sum of a random number of geometric random variable, but still I can't go any further beyond that. Thank you!

• For $i=0,1,2,3$, let $E_i$ be the expected number of days until $3$ consecutive rainy days, if the chain is currently in state $i$. We are asked to compute $E_0$. Notice that $E_0=1+.3E_2$. Develop similar equations for the other $E_i$ and solve. Jun 29, 2021 at 17:39
• Developing the equations is where I am stuck at and I am also confused as to whether there is a boundary condition. Could you please explain to me, for instance, how to develop E0? Jun 29, 2021 at 18:44

Let me get you started.

I claim that $$E_0=1+.3E_2$$ as I stated in a comment.

Suppose we are in state $$0$$. We must wait at least $$1$$ more day to see if we'll have three consecutive days of rain. $$70\%$$ of the time, it rains, and we are done, but $$30\%$$ of the time we transition to state $$2$$, and then we must wait, on average, $$E_2$$ days to get $$3$$ consecutive rainy days.

If we are in state $$2$$, similar reasoning gives $$E_2=1+.5E_1+ .5E_3$$

You can read the equations right off your diagram. You should get $$4$$ equations with $$4$$ unknowns and a unique solution. I'm sure you won't have any problem finishing from here.

• I have found E3=1 + 0.6(E3) + 0.4(E1) (state 1 is the only different state we can transition to from state 3 and we might also stay in state 3 with p=0.6?) and E1= 1 + 0.5(E0) + 0.5(E2). Do you think I made a mistake or those are correct? Jun 30, 2021 at 8:20
• @SydneyAstbury I think you are correct. Jun 30, 2021 at 11:36

Ordinarily, Markov Chains are conditional on the previous step, but not on the previous two steps. A way to get around this in the current problem is to re-define the states to account for two days, with suitable overlaps.

The new states are 00 (for consecutive dry days) 01 (dry followed by wet), and so on to 11 (for two wet days in a row).

States 'overlap' so that 00 can be followed by 01, but not by 01, etc. In the matrix below we call the new states A,B, C, D.

The transition matrix can be written in R as

P = matrix(c(.6, .4,  0,  0,
0,  0, .5, .5,
.5, .5,  0,  0,
0,  0, .3, .7), byrow=T, nrow=4)


We seek $$\sigma$$ such that $$\sigma P = P,$$ so that $$\sigma$$ is a left eigenvector of $$P.$$ Because R finds right eigenvectors, we use the transpose t(P).

The stationary distribution is proportional to the right eigenvector of smallest modulus, which R prints first. [In R, the symbol %*% denotes matrix multiplication.]

g = eigen(t(P))\$vec[,1]
sg = g/sum(g); sg
[1] 0.2542373 0.2033898 0.2033898 0.3389831
sg %*% P  # check
[,1]      [,2]      [,3]      [,4]
[1,] 0.2542373 0.2033898 0.2033898 0.3389831


So the long-run probability of rain is $$0.5424.$$

Note: In this relatively simple problem it is not difficult to solve a few simultaneous equations, but the eigen-method shown above is very convenient for ergodic chains with more than four states.