Markov Chain Question - Request help finishing the question

Question

Sorry for the question but for some reason I'm just unable to see the answer to this. some help would be greatly appreciated.

The weather on any given day is either sunny, cloudy or rainy...We let $\{X_{t} \}_{t \in \mathbb{N}}$ be the markov chain denoting the state of the weather at time t.

Using the information in the question I've derived the one-step transition matrix $$P = \begin{pmatrix}0.5 & 0.5 & 0 \\ 0.4 & 0.4 & 0.2 \\ 0& 0.5 & 0.5\end{pmatrix}$$ for sunny/cloudy/rain respectively. we have the stationary distribution: $\pi = (4/11,5/11,2/11)$

i'm now asked:

Alice carries an umbrella with her if the day is either cloudy or rainy. How likely is it that Alice is carrying an umbrella today given that (i) she carried an umbrella yesterday, (ii) she carried an umbrella the last two days?

and for some reason i have an utter mental block. i understand that these two parts have the same answer, given the markov property doesn't really care. but i cant seem to be able to calculate alice carrying the umbrella.

In my head i need to calculate the probability Cloudy given Cloudy, Rain given Rain, rain given cloudy and cloudy given rain... but this also seems wrong to me and so just generally unsure of where to go on from..

please put me out of my misery and send aid... I thank you for taking the time to read this.

Answer 1

Alice carrying an umbrella on day $\ d\ $ is the same event as that the Markov chain is in either state $\ 2\ $ or state $\ 3\ $ on day $\ d\ $. So the probability that Alice is carrying an umbrella today, given that she carried one yesterday is $$ P\big(X_0=3\big| X_{-1}=3\ \text{ or }\ X_{-1}=4 \big)+ P\big(X_0=4\big| X_{-1}=3\ \text{ or }\ X_{-1}=4 \big)\\ =\frac{P\big(X_0=3,X_{-1}=3\big)+ P\big(X_0=3,X_{-1}=4\big)}{P\big(X_{-1}=3\big)+ P\big(X_{-1}=4\big)}\\ + \frac{P\big(X_0=4,X_{-1}=3\big)+ P\big(X_0=4,X_{-1}=4\big)}{P\big(X_{-1}=3\big)+ P\big(X_{-1}=4\big)}\\ =\frac{\displaystyle\sum_{i=3}^4\sum_{j=3}^4 \pi(-1)_i P_{ij}}{\pi(-1)_3+\pi(-1)_4}\ , $$ Similarly, the probability that Alice is carrying an umbrella today, given that she carried one on the last two days is $$ \frac{\displaystyle\sum_{i=3}^4\sum_{j=3}^4 \sum_{k=3}^4\pi(-2)_i P_{ij}P_{jk}}{\displaystyle\sum_{i=3}^4\sum_{j=3}^4\pi(-2)_iP_{ij}}\ . $$ To evaluate these quantities, you need to know the state distribution on the preceding two days, which you can presumably assume to be the stationary distribution. Your intuition that these two quantities should be equal, however, has led you astray. They aren't.