# Markov Process: predict the weather using a stochastic matrix

I have the following stochastic matrix

$$P = \begin{pmatrix} P(S \mid S) = 0.5 & P(F \mid S) = 0.2 & P(R \mid S) = 0.3 \\ P(S \mid F) = 0.2 & P(F \mid F) = 0.7 & P(R \mid F) = 0.1 \\ P(S \mid R) = 0.75 & P(F \mid R) = 0.15 & P(R \mid R) = 0.1 \end{pmatrix}$$

Where $S$ means sunny, $R$ means rainy and $F$ means foggy. For example $P(S \mid F)$ is the probability of being sunny tomorrow given that today is foggy. Similarly, $P(R \mid R)$ is the probability of tomorrow being rainy if today is also rainy.

The graph representing the situation would look like this

Now suppose today is Monday and it's sunny. I would like to estimate the probabilities of

1. a sunny Tuesday

2. a sunny Tuesday and Wednesday

3. a sunny Wednesday

On Wikipedia there's a similar problem, but since I am new to these things, I decided to ask you help you anyway at least for some clarifications.

As far as I have understood a Markov process is a memoryless stochastic process, that only remembers the current state in order to predict future states.

For point $1$, we can simply use the fact that $P(S \mid S) = 0.5$, i.e. the probability of tomorrow being sunny if today is sunny is $0.5$, which is therefore the answer.

For the other cases it's a little bit more complicated: we cannot use directly any value from the stochastic matrix $P$. We need of course to do intermediary calculations.

The example I am linking to, to predict the weather on day $1$, it creates a vector representing the weather on day $0$. Similarly, in my case, I think that vector would be $$v_1 = \begin{pmatrix} 1 & 0 & 0 \end{pmatrix}$$ that is the $1$ represents that it's $100 \%$ sunny. If I multiply $v_1$ by $P$ I obtain (similarly to the Wikipedia's problem above) the following vector $$v_1 =\begin{pmatrix} 0.5 & 0.2 & 0.3 \end{pmatrix}$$

From this to obtain my guess of $0.5$ there's some distance of reasoning. Could you please explain it to me

1. How do you explain the $0.5$ I am guessing (if it's correct) from the calculations that I just did and done in the solutions to the Wikipedia's problem. If it's not correct, why and how can I solve it?

2. In the third question, do I need to do the same thing as I did for the first point, but using $P^2$, like in the Wikipedia's article?

3. The second question involves a compound probability (I think). Am I right? How exactly would I proceed to solve this case, and why?

Thanks for any help (and sorry for the long post)!

The way you have written out the matrix $P$, we pre-multiply them with a row vector, and obtain another row vector that represents the next day's weather distribution. In general, we say

$$v_{i+1} = v_i P$$

where $v_i$ represents the $i$th day's weather distribution. In this case, we have $v_1 = \left[ \begin{array}{ccc} 1 & 0 & 0 \end{array} \right]$, and then

\begin{align} v_2 & = v_1 P \\ & = \left[ \begin{array}{ccc} 1 & 0 & 0 \end{array} \right] \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \\ 0.2 & 0.7 & 0.1 \\ 0.75 & 0.15 & 0.1 \end{array} \right] \\ & = \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \end{array} \right] \end{align}

which tells us that Tuesday will be sunny with probability $0.5$.

We could, as you suggest, do the same thing with $P^2$ to obtain Wednesday's weather distribution, but it might be easier simply to apply $P$ to $v_2$:

\begin{align} v_3 & = v_2 P \\ & = \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \end{array} \right] \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \\ 0.2 & 0.7 & 0.1 \\ 0.75 & 0.15 & 0.1 \end{array} \right] \\ & = \left[ \begin{array}{ccc} 0.515 & 0.285 & 0.2 \end{array} \right] \end{align}

which tells us that Wednesday will be sunny with probability $0.515$.

It is simplest to answer the middle question—the probability that it will be sunny both Tuesday and Wednesday—by simply observing that it is sunny on Tuesday with probability $0.5$ (as we already observed), and then identical reasoning tells us that conditioned on that fact, it will also be sunny on Wednesday with probability $0.5$, and multiply those together to obtain $0.25$.

Alternatively, one could eliminate the rainy and foggy probabilities from $v_2$, and inspect the first component of

$$\left[ \begin{array}{ccc} 0.5 & 0 & 0 \end{array} \right] \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \\ 0.2 & 0.7 & 0.1 \\ 0.75 & 0.15 & 0.1 \end{array} \right] = \left[ \begin{array}{ccc} 0.25 & 0.1 & 0.15 \end{array} \right]$$

to obtain the same answer. Note that this "clipped" vector is not a probability distribution (it does not sum to $1$), but that does not matter for our present purposes.

• You say: "and then identical reasoning tells us that conditioned on that fact, it will also be sunny on Wednesday with probability 0.5", you mean $0.515$ instead of this $0.5$, right? Or am I missing something? – nbro Dec 15 '15 at 19:18
• No, I mean what I wrote, I'm pretty sure. The probability that it's sunny on Tuesday given that it's sunny on Monday is $0.5$, and the probability that it's sunny on Wednesday given that it's sunny on Tuesday is also $0.5$ (by identical reasoning), so the probability that it's sunny on both Tuesday and Wednesday, given that it's sunny on Monday, is $0.5 \times 0.5 = 0.25$. – Brian Tung Dec 15 '15 at 19:20
• Ok. So, the initial matrix helps us also in other reasonings, and not necessarily on the relationship between the possible initial day and its successive (of course?). Apart from that, you also multiply $0.5$ by $0.5$, but aren't the events dependent? Yes, indeed! (?) – nbro Dec 15 '15 at 19:24
• It doesn't matter whether they're independent, because you're using the definition of conditional probability: $P(A, B) = P(B \mid A) P(A)$. In this case, $A$ is "it's sunny on Tuesday" and $B$ is "it's sunny on Wednesday". (Further, all of this is conditioned on it being sunny on Monday, but that doesn't affect the multiplication.) – Brian Tung Dec 15 '15 at 21:02
• Yeah, it seems to make sense! Thanks! – nbro Dec 15 '15 at 22:21