0
$\begingroup$

Markov chains are widely used in modeling several natural and social processes. Consider the following three-state Markov chain modeling the daily weather in Boston. Each day can be sunny, partly cloudy, or rainy. The Markov chain gives you the probability that the next day is sunny, partly cloudy, or rainy, given the state of the current day. For instance, if today is sunny, then the probability that tomorrow is sunny is 0.5, the probability that tomorrow is partly cloudy is 0.4, and the probability that tomorrow is rainy is 0.1.

enter image description here

Analyze the above Markov chain to answer the following questions. For the last three parts, you need to solve the system of linear equations that give the stationary distribution of the Markov chain.

If today is rainy, what is the probability that the next two days are also rainy? /100

If today is rainy, what is the probability that the day after tomorrow is sunny? /100

In the stationary distribution, what is the probability of any day being sunny? /100

In the stationary distribution, what is the probability of any day being partly cloudy? /100

In the stationary distribution, what is the probability of any day being rainy? /100

I understand how it works with the probability of the next day, but i'm having trouble with the day after. I'm not sure how you would calculate the probability for the next two days since the chain shows only the next day?

$\endgroup$
  • $\begingroup$ Do you know about the transition matrix already? To handle two days in the future you will ultimately need to calculate the square of the transition matrix (albeit not necessarily in matrix form in this small example). $\endgroup$ – Ian Mar 26 '15 at 14:47
  • $\begingroup$ How would I find the transition matrix? $\endgroup$ – smd Mar 26 '15 at 15:17
0
$\begingroup$

The transition matrix $P$ has entries $p_{ij}$ which are the probability of transitioning from $i$ to $j$. So if your states are listed as "sunny,partly cloudy,rainy" then

$$P=\begin{bmatrix} 0.5 & 0.4 & 0.1 \\ 0.4 & 0.5 & 0.1 \\ 0.2 & 0.2 & 0.6 \end{bmatrix}$$

Given an initial distribution $\lambda$ (written as a row vector), the distribution at time $n$ is $\lambda P^n$. (If you prefer, you can work with a column vector and deal with $(P^T)^n \lambda$.) So you can find your second answer by taking $\lambda=[0,0,1]$ and calculating $\lambda P^2$.

The first answer is slightly different because you are being asked about the probability that it rains two days in a row. Here you'll want to use the independence: the probability of being rainy for the next two days is the probability to go from rainy to rainy, and then rainy to rainy again.

$\endgroup$
  • $\begingroup$ I got the first one, but I'm still having trouble figuring out how to get the other answers. For the second one, how would I know the probability of the day after tomorrow if i don't know tomorrow? $\endgroup$ – smd Mar 26 '15 at 19:00
  • $\begingroup$ @smd The formula with the transition matrix is really telling you to use a weighted average of the probabilities of being sunny in 2 days, given each of the possible weathers for tomorrow. Here the weights are the probabilities of each weather tomorrow. $\endgroup$ – Ian Mar 26 '15 at 19:30
1
$\begingroup$

P=\begin{bmatrix} 0.5 & 0.4 & 0.1 \\ 0.4 & 0.5 & 0.1 \\ 0.2 & 0.2 & 0.6 \end{bmatrix} This transition matrix represents the current days conditions as the rows and tomorrows condition as the column. So to find the probability of a rainy tomorrow given today is sunny go to $P_{13}$. To find future days you must multiply the matrix times itself. $P^2$ would represent the conditions in two days.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.