need help with a Markov chain for weather conditions This is my second time posting the question as I failed to do so the first time, because I did not know the proper way. My apologies.
Question:
Suppose that whether or not it rains today depends on weather conditions through the
previous teo days. If it has rained for the past two days, then it will rain today with
probability 0.7. If it did not rain for any of the past two days, then it will rain today with
probability 0.4. In any other case the weather today will, with probability 0.5, be the same
as the weather yesterday. Describe the weather condition over time with a Markov chain.
Suppose that it rained today and yesterday. Calculate the expected number of days until it rains three consecutive days for the first time in the future.
I have found 4 different states that I named RR(0), RN(2), NR(1), and NN(3). R stands for when it rains and N is for when it does not.
As the question asks, I have tried finding the possible ways of three consecutive days being rainy.
At time n, we are given it was rainy today and yesterday, meaning we are in State 0.
1-) First possibility is when it rains tomorrow, which gives us RRR (we got the three consecutive days)
2-) Second possibility is when we go from 0 to 2, from 2 to 1, from 1 to 0, and stay in 0 one day. That follows as : RRNRRR (In 4 days we can get rain for 3 consecutive days)
3-) Third is when we go from 0 to 2, from 2 to 3, from 3 to 1, from 1 to 0, and stay in 0 one day. That follows as: RRNNRRR.
To conclude what I have in mind about the question, wherever we go, when we get to State 0, we need to stay there one more day to get three consecutive rainy days. That means the minimum # of days to get rain for three consecutive days is one day. However, after this point, I am not able to proceed with the question.
Any help would be appreciated, thank you!!
Edit: I think the maximum # of days is just a random number, which leads me to the expectation of the sum of a random number of geometric random variable, but still I can't go any further beyond that. Thank you!
 A: Let me get you started.
I claim that $$E_0=1+.3E_2$$ as I stated in a comment.
Suppose we are in state $0$.  We must wait at least $1$ more day to see if we'll have three consecutive days of rain.  $70\%$ of the time, it rains, and we are done, but $30\%$ of the time we transition to state $2$, and then we must wait, on average, $E_2$ days to get $3$ consecutive rainy days.
If we are in state $2$, similar reasoning gives $$E_2=1+.5E_1+ .5E_3$$
You can read the equations right off your diagram.  You should get $4$ equations with $4$ unknowns and a unique solution.  I'm sure you won't have any problem finishing from here.
A: Ordinarily, Markov Chains are conditional on the previous step, but not on the previous two steps.
A way to get around this in the current problem is to re-define the
states to account for two days, with suitable overlaps.
The new states are 00 (for consecutive dry days) 01 (dry followed by wet), and so on to
11 (for two wet days in a row).
States 'overlap' so that 00 can be followed by 01, but not by 01, etc. In the matrix below
we call the new states A,B, C, D.
The transition matrix can be written in R as
P = matrix(c(.6, .4,  0,  0,
              0,  0, .5, .5,
             .5, .5,  0,  0,
              0,  0, .3, .7), byrow=T, nrow=4) 

We seek $\sigma$ such that $\sigma P = P,$ so
that $\sigma$ is a left eigenvector of $P.$
Because R finds right eigenvectors, we use the
transpose t(P).
The stationary distribution is proportional to
the right eigenvector of smallest modulus, which R
prints first. [In R, the symbol %*% denotes matrix multiplication.]
g = eigen(t(P))$vec[,1]
sg = g/sum(g); sg
[1] 0.2542373 0.2033898 0.2033898 0.3389831
sg %*% P  # check
          [,1]      [,2]      [,3]      [,4]
[1,] 0.2542373 0.2033898 0.2033898 0.3389831

So the long-run probability of rain is $0.5424.$
Note: In this relatively simple problem it is not difficult to solve a few simultaneous equations, but the eigen-method shown above is very convenient for ergodic chains with more
than four states.
