# Markov Chains Question

Markov chains are widely used in modeling several natural and social processes. Consider the following three-state Markov chain modeling the daily weather in Boston. Each day can be sunny, partly cloudy, or rainy. The Markov chain gives you the probability that the next day is sunny, partly cloudy, or rainy, given the state of the current day. For instance, if today is sunny, then the probability that tomorrow is sunny is 0.5, the probability that tomorrow is partly cloudy is 0.4, and the probability that tomorrow is rainy is 0.1.

Analyze the above Markov chain to answer the following questions. For the last three parts, you need to solve the system of linear equations that give the stationary distribution of the Markov chain.

If today is rainy, what is the probability that the next two days are also rainy? /100

If today is rainy, what is the probability that the day after tomorrow is sunny? /100

In the stationary distribution, what is the probability of any day being sunny? /100

In the stationary distribution, what is the probability of any day being partly cloudy? /100

In the stationary distribution, what is the probability of any day being rainy? /100

I understand how it works with the probability of the next day, but i'm having trouble with the day after. I'm not sure how you would calculate the probability for the next two days since the chain shows only the next day?

• Do you know about the transition matrix already? To handle two days in the future you will ultimately need to calculate the square of the transition matrix (albeit not necessarily in matrix form in this small example). – Ian Mar 26 '15 at 14:47
• How would I find the transition matrix? – smd Mar 26 '15 at 15:17

The transition matrix $P$ has entries $p_{ij}$ which are the probability of transitioning from $i$ to $j$. So if your states are listed as "sunny,partly cloudy,rainy" then
$$P=\begin{bmatrix} 0.5 & 0.4 & 0.1 \\ 0.4 & 0.5 & 0.1 \\ 0.2 & 0.2 & 0.6 \end{bmatrix}$$
Given an initial distribution $\lambda$ (written as a row vector), the distribution at time $n$ is $\lambda P^n$. (If you prefer, you can work with a column vector and deal with $(P^T)^n \lambda$.) So you can find your second answer by taking $\lambda=[0,0,1]$ and calculating $\lambda P^2$.
P=\begin{bmatrix} 0.5 & 0.4 & 0.1 \\ 0.4 & 0.5 & 0.1 \\ 0.2 & 0.2 & 0.6 \end{bmatrix} This transition matrix represents the current days conditions as the rows and tomorrows condition as the column. So to find the probability of a rainy tomorrow given today is sunny go to $P_{13}$. To find future days you must multiply the matrix times itself. $P^2$ would represent the conditions in two days.