So I understand what it means for a stochastic process to have the Markov property, but I am not as sure about how to actually deduce this from information given.

For example,

Suppose a washing machine can serve people one at a time, moreover, if it is washing a persons clothes at time n, it has probability $p$ of finishing the cycle before time $n+1$.

Between the times $n$ and $n+1$, $P_{n}$ customers arrive, where $P_{n}$ is $Poisson(\lambda)$ independent of past arrivals.

Show that the total number of customers waiting service and currently being served at time n is a Markov chain.

So how could I show that




Would the transition probabilities best be found first, or could they be deduced once we have formulated it?

One thing I am a bit confused about is,

Between times $n$ and $n+1$, can the washing machine only serve one customer?

Ie, it says it serves one customer at a time, is this implying at most 1 person could leave between these times?

Assuming that this is the case,

At time $n$ we have $X_{n}$ total customers, between times $n$ and $n+1$ , $P_{n}$ customers arrive, and either 0 customers leave or one leaves, with probability p.

Ie, $X_{n+1}=X_{n}+P_{n}-1$ with probability $p$ and $X_{n+1}=X_{n}+P_{n}$ with probability $(1-p)$

However, as suggested in the comments, I am still not exactly sure how that implies we can write

$X_{n+1}=X_{n}+P_{n}-B_{n}$ where $B_{n}$ is Bernoulli, because what does the $n$ have to do with it?

I tried breaking it into cases,

in any time period at most one person leaves, and $P_{n}$ arrive,

so either no people leave or one person leaves.


$X_{n+1}=X_{n}+P_{n}-1$ with probability p and

$X_{n+1}=X_{n}+P_{n}$ with probability $1-p$

Now, can I use that both the Bernoulli trials and Poisson random variable are respectively independent of past to make the conclusion?

I would also like to understand how it all relates to the transition probabilities for every (i,j) ie how that can be found from the information I am still pretty confused about this question and am looking for any solid explanations Thanks

  • $\begingroup$ Notice that $X_{n+1} = X_{n} + P_n - B_n$, where $B_n$ has Bernoulli distribution with probability $p$ and $B_n$ is independent from the past. $\endgroup$ – echzhen Feb 19 '17 at 1:25
  • $\begingroup$ @echzhen So the total number of people at time n+1 would be the number of people at time n + the random $P_{n}$ , but what would the $-{Bn}$ represent? Because only one person could leave between time $n $and $n+1$, no? $\endgroup$ – Quality Feb 19 '17 at 1:51
  • $\begingroup$ if I understood the problem correctly: between $n+1$ and $n$ only one person can leave independently from the past. So, yes $B_n$ represents the person who is currently being served. $\endgroup$ – echzhen Feb 19 '17 at 12:57
  • $\begingroup$ Slight correction: $X_{n+1}=(X_n+P_n-B_n)^+$, the positive part being needed when $X_n=P_n=0,B_n=1$. $\endgroup$ – John Dawkins Feb 19 '17 at 16:46
  • $\begingroup$ @JohnDawkins Is that the case that there are zero people waiting, zero people arrive and the person being served is finished and leaves? $\endgroup$ – Quality Feb 19 '17 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.