0
$\begingroup$

I'm giving my first steps in stochastic processes but I'm having some difficulties. See the following example

Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability 0.7; if it rained today but not yesterday, then it will rain tomorrow with probability 0.5; if it rained yesterday but not today, then it will rain tomorrow with probability 0.4; if it has not rained in the past two days, then it will rain tomorrow with probability 0.2.

If we let the state at time n depend only on whether or not it is raining at time n, then the preceding model is not a Markov chain (why not?). However, we can transform this model into a Markov chain by saying that the state at any time is determined by the weather conditions during both that day and the previous day. In other words, we can say that the process is in

state 0 if it rained both today and yesterday,

state 1 if it rained today but not yesterday,

state 2 if it rained yesterday but not today,

state 3 if it did not rain either yesterday or today.

$$P=\begin{bmatrix}0.7&&0&&03&&0\\0.5&&0&&0.5&&0\\0&&0.4&&0&&0.6\\0&&0.2&&0&&0.8\end{bmatrix}$$

I'm wondering how this matrix is computed, $$P_{ij}=P(state_i|state_j)$$ I am unable to see clearly how these conditionals are calculated.

$\endgroup$
1
$\begingroup$

Denoting the current day as index $i$ (so that yesterday is $i-1$ and the day before yesterday is $i-2$), the previous state comprises two elements, $State_{n-1}=\{S_{i-1},S_{i-2}\}$, where $S_k$ is the state of the weather for day $k$. Thus the current state is $State_{n}=\{S_{i},S_{i-1}\}$

Lettting $R$ be the occurrence of rain on a particular day, so that $\overline{R}$ is when no rain occurs for that day, there are $4$ possible values of the previous state $State_{n-1}$ $$\{\overline{R},\overline{R}\}, \{\overline{R},R\},\{R,\overline{R}\},\{R,R\}$$ For each of these $4$ states, there are only two possible next states:- $$\begin{align} State_n=\{\overline{R},\overline{R}\}\text { or }State_n=\{R,\overline{R}\}\text{ given }State_{n-1}=\{\overline{R},\overline{R}\} \\ State_n=\{\overline{R},\overline{R}\}\text { or }State_n=\{R,\overline{R}\}\text{ given }State_{n-1}=\{\overline{R},R\} \\ State_n=\{\overline{R},R\}\text { or }State_n=\{R,R\}\text{ given }State_{n-1}=\{R,\overline{R}\} \\ State_n=\{\overline{R},R\}\text { or }State_n=\{R,R\}\text{ given }State_{n-1}=\{R,R\} \end{align}$$ With the information given in the question, and based on the constraints of what the next state is based on the current state, you can figure out the structure of the transition matrix.

For example, the information Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$, corresponds to $$P(State_{n+1}=\{\overline{R},\overline{R}\}|State_{n}=\{\overline{R},\overline{R}\})=0.7\\\Rightarrow P(State_{n+1}=\{R,\overline{R}\}|State_{n}=\{\overline{R},\overline{R}\})=1-0.7=0.3$$ and if it rained today but not yesterday, then it will rain tomorrow with probability 0.5 corresponds to $$P(State_{n+1}=\{\overline{R},\overline{R}\}|State_{n}=\{\overline{R},R\})=0.5\\\Rightarrow P(State_{n+1}=\{R,\overline{R}\}|State_{n}=\{\overline{R},R\})=1-0.5=0.5$$

$\endgroup$
  • $\begingroup$ I just do not understand how conditional probabilities are in fact calculated $\endgroup$ – Roland Jul 5 '15 at 17:38
  • 1
    $\begingroup$ @JohnSheridan I have added a few more lines to my answer to help clarify. $\endgroup$ – Alijah Ahmed Jul 5 '15 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.