Discrete time queue markov chain Consider  a  queue  at  a  counter,  in  which $X_n$ denotes the number of customers who are either waiting or being served at time $n$.Suppose $Y_{n+1}$ new customers join the queue between $n$ and $n+1$, and whenever $X_n > 0$, $Z_{n+1}$ customers are processed and leave the queue.
(a) Suppose we assume that $X_0,Y_1, Z_1, Y_2, Z_2,...$, are all mutually independent.  Assume in addition that ${Y_i}$ are independent and identically distributed Bernoulli $(p)$ random variables for some $p∈(0,1)$, and suppose that ${Z_i}$ are independent  and  identically  distributed  Bernoulli  $(q)$ random  variables for some $q∈(0,1)$, with $p <1$.  Find the stationary distribution $π$ for the Markov chain ${X_n}$.  Is it unique?
(b)Compute $E_π[X_n]$, where $E_π$ denotes the expectation of $X_n$ given that the initial distribution of the chain $X_0$ is given by the stationary distribution $π$ computed in (a).
What I have so far is that $X_{n+1} = X_n + Y_{n+1} - l_{X_n>0}Z_{n+1}$, where l is the indicator function. In a stationary distribution $π$ for Markov Chain $X_n$, $π = π*P$. I am given that $Y_i$ and $Z_i$ are Bernoulli random variables, but I am unsure how to put together this information to create the transition probability matrix $P$ and then find the matrix $π$ made invariant by $P$. 
For part b, I am not too sure how to find $E_π[X_n]$. I was thinking that because it is invariant, I could turn $X_{n+1} = X_n + Y_{n+1} - l_{X_n>0}Z_{n+1}$ into $X_{n} = X_{n-1} + Y_{n} - l_{X_{n-1}>0}Z_{n}$ and then find $E[X_{n} = X_{n-1} + Y_{n} - l_{X_{n-1}>0}Z_{n}]$.Because I know that $Y$ and $Z$ are mutually independent, We can do: $E[X_{n}] = E[X_{n-1}] + E[Y_{n}] - El_{X_{n-1}>0}Z_{n}]$. So, $E[X_{n}] = E[X_{n-1}] + p - El_{X_{n-1}>0}Z_{n}]$. I know that the $E[Z_n] = q$, but I don't know how to simplify from what I currently have or what $E[X_{n-1}]$ is. 
 A: Your very first goal should be to turn this description into a Markov chain. You know the states of the Markov chain: at any time, there can be $0, 1, 2, 3, \dots$ customers in the line. So it remains to figure out the transition probabilities.
Suppose that at step $n$ there are $X_n=k$ customers in the line. Then at step $n+1$, three things can happen:


*

*With probability $p(1-q)$, $Y_{n+1} = 1$ and $Z_{n+1}=0$: one customer joins the line and none leave. So at step $n+1$ there will be $X_{n+1} = k+1$ customers in the line.

*With probability $(1-p)q$, $Y_{n+1} = 0$ and $Z_{n+1}=1$: no new customers arrive, but one customer leaves. So at step $n+1$ there will be $X_{n+1} = k-1$ customers in the line.

*Otherwise, either nothing happens, or one customer arrives and another leaves; in either case, $X_{n+1} = k$.


Only if $X_n = 0$ do we do something else. In this case, we simply have a probability $p$ of having $X_{n+1} = 1$, and a probability $1-p$ of having $X_{n+1} = 0$.
Altogether, the Markov chain looks like:

(I have left the loops unlabeled; at every state, there's some probability of staying in that state which can be inferred from the probability of leaving it.)
Now you can proceed with actually solving the problem. You could write down the transition probabilities in the variables $\pi_0, \pi_1, \pi_2, \dots$; there will be infinitely many equations, but almost all of them will be very similar.
An alternate approach is to use the time-reversibility equations. In these case, these say that the fraction of transitions going from state $k$ to $k+1$ (which is $\pi_k$ times the transition probability $P_{k,k+1}$) is equal to the fraction of transitions going from state $k+1$ to $k$ (which is $\pi_{k+1}$ times the transition probability $P_{k+1,k}$). Solving these is much more straightforward: you immediately get an expression for $\pi_1$ in terms of $\pi_0$, $\pi_2$ in terms of $\pi_1$, and so on.
Finally, once you've solved for the stationary distribution, you should have
$$
   \mathbb E_{\pi}[X_n] = \sum_{k=0}^\infty k \cdot \pi_k.
$$
This does not depend on $n$: if you start by sampling $X_0$ from the stationary distribution $\pi$, then $X_1, X_2, \dots$ should also all follow the stationary distribution.
