# How to get the proportion of time of a state of this markov chains process

Suppose each day there is a $0.2$ probability will rain in the morning. $P(\text{rains afternoon}|\text{rain morning})=0.6$ and $P(\text{rains afternoon}|\text{not rain morning})=0.3$. Suppose John would go to the office in the morning and leave in the afternoon and he totally get 3 umbrellas. Let $X_n$ be the number of umbrellas storing in his office at $nth$ night. Whats the proportion of time he has $0$ umbrella in his office at night?

I tried to make the transition probability matrix and find the stationary distribution at state $0$ and found that $\pi_0=0.19$ but seems not consistent with the answer but i don't know what is the mistake. Any hints or solution are welcome.

$P_{transition} =\begin{pmatrix}0.92&0.08&0&0\\0.24&0.68&0.08&0\\0&0.24&0.68&0.08\\0&0&0.24&0.76\end{pmatrix}$ so $P-I=\begin{pmatrix}-0.08&0.08&0&0\\0.24&-0.32&0.08&0\\0&0.24&-0.32&0.08\\0&0&0.24&-0.24\end{pmatrix}$

-
In a transition matrix, the entries in each column must add up to $1$ (or, if you are using the other convention, the entries in each row must add up to $1$). So have another think about how to set up the transition matrix. –  Gerry Myerson Dec 17 '12 at 5:44
O i made some mistake, –  Mathematics Dec 17 '12 at 5:49
@GerryMyerson Ya i think you are right, i needa compute again –  Mathematics Dec 17 '12 at 5:55
@GerryMyerson have write in the answer space, is it correct? –  Mathematics Dec 17 '12 at 6:19

It would help a lot if you gave some indication as to how you got $0.19$ --- then maybe someone would see an arithmetical mistake, and we could all go home. Since you haven't done that, and since I'm not inclined to do the whole problem, I'll just indicate how one might get proceed.

$X_n$ goes up by one when it rains in the morning and not in the afternoon. The probability of this is $.08$ (unless $X_n=3$, in which case the probability is zero).

$X_n$ goes down by one when it doesn't rain in the morning but does rain in the afternoon. The probability of this is $.24$ (unless $X_n=0$, in which case the probability is zero).

This gives you all the transition probabilities; you can now figure out the eigenvector for the eigenvalue $1$ and thus the answer to the question.

-
I have provided some step for your calculation, is it the same as your answer? –  Mathematics Dec 17 '12 at 5:23

$P_{transition} =\begin{pmatrix}0.92&0.08&0&0\\0.24&0.68&0.08&0\\0&0.24&0.68&0.08\\0&0&0.24&0.76\end{pmatrix}$ so $P-I=\begin{pmatrix}-0.08&0.08&0&0\\0.24&-0.32&0.08&0\\0&0.24&-0.32&0.08\\0&0&0.24&-0.24\end{pmatrix}$ $\pi_0=3\pi_1=9\pi_2=27\pi_3$ and so $\pi_0=0.675$

-
Looks good to me. –  Gerry Myerson Dec 18 '12 at 5:37