Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I want to describe this case in markov chain:

The case:
Mr. Meier reads NYTimes everyday and puts the newspaper on news rack. His wife sometimes cleans the house(with prob $1/3$ each day) and throws away all newspapers. If there are 5 or more newspapers on the rack, Mr. Meier throws away them by his own hand. Now lets assume, husband and wife are gone to bed. How can i describe the number of newspapers on the news rack with markov chain?

I am kinda stuck, what are the my state spaces here?

share|improve this question
The state space is $[0;5]$ - that is, how many NYT newspapers can be at the rack. In general it shall be $\Bbb N_0$, but you know that $p_{i0} = 1$ for all $i\geq 5$, so you can lump all the states bigger than $4$ in one. I didn't understand, though, what's the order of cleaning/throwing. –  Ilya Jul 18 '13 at 7:49
@Ilya, yeaah, you are right, state space are # of newspapers. so it is from 0 to 4, beause if =5 or >5, Mr.Meier throws all away. i will give you a link of what i drawn in a second –  doniyor Jul 18 '13 at 7:53
@Ilya, imagebin.org/264899, this is what i saw in solution, but i dont understand why $2/3$. can you please take a look? –  doniyor Jul 18 '13 at 7:56
@Ilya, i think, now i got it. 2/3 is the prob that wife DOES NOT clean the house, so that #of newspapers can grow. am i right? –  doniyor Jul 18 '13 at 7:59
Ok, $p_{i,i+1} = \frac23$ for $i=0,1,2,3$ (the case when the wife does not clean) and $p_{i,0} = \frac13$ for $i=0,1,2,3$ (the case when the wife does clean). Finally, $p_{4,0} = 1$ thanks to Mr. Meier. –  Ilya Jul 18 '13 at 8:10

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.