# Markov chains and queues

I do not understand how may I use the Markov Chain $Y$ and and describe the system $X$ using the states that the exercise suggest. I was searching queue's examples and -i understand this is a M/M/1 queue but I'm stuck in how I use the $Q$-matrix to describe the system. The new state space are the couples $(A_k,B_k)$? . Thanks for your support.

Customers arrive at a certain queue in a Poisson process of rate $\lambda$. The single 'server' has two states $A$ and $B$, state $A$ signifying that he is 'in attendance' and state $B$ that he is having a tea-break. Independently of how many customers are in the queue, he fluctuates between these states as a Markov chain $Y$ on $\{A,B\}$ with $Q$ -matrix \begin{equation*} \begin{pmatrix} -\alpha & \alpha \\ \beta & -\beta\\ \end{pmatrix} \end{equation*} The total service time for any costumer is exponentially distributed with parameter $\mu$ and is independent of the chain $Y$ and of the service times of other customers. Describe the system as a Markov chain $X$ with the state-space \begin{equation*} \{A_0,A_1,A_2,\dots\} \cup\{B_0,B_1,B_2,\dots\}, \end{equation*} $A_n$ signifying that the server is in state $A$ and there are $n$ people in the queue (including anyone being served) and $B_n$ signifying that the server is in state $B$ and there are $n$ people in the queue.

## 1 Answer

No, the state space is exactly as they described it; it is the union of the states $\{A_0, A_1, A_2, \ldots\}$, where the server is on, and the states $\{B_0, B_1, B_2, \ldots\}$, where the server is off. Alternatively, you could view the states as the ordered pairs $(\delta, i)$, where $\delta = 0$ if the server is off and $1$ if it is on, and $i$ means that that there are $i$ customers in the system. Then, $(0, i)$ maps to $B_i$ and $(1, i)$ maps to $A_i$.

The dynamics of the system are, presumably, that one goes from state $A_i$ to $A_{i+1}$ at rate $\lambda$, to state $A_{i-1}$ at whatever the service rate is ($\mu$?), and to state $B_i$ at rate $\alpha$. One goes from state $B_i$ to $B_{i+1}$ also at rate $\lambda$, and to state $A_i$ at rate $\beta$. (There is no transition to state $B_{i-1}$, because no customers are served if the server is off.)

ETA: To be clearer, I envision the system depicted below. I left the service transitions (i.e., from $A_{i+1}$ to $A_i$, for $0 \leq i$) unlabelled because we don't know whether service rate varies by number of customers, but if they don't, then all those transitions can be labelled with some service rate $\mu$.

• maybe I'm more confused. Now I understand that I have two 'process' happening (attendance and tea-break) and they switch at rate $\alpha$ and $\beta$. It will be more easy to me if you explain using kind of transition matrix (I'm a beginner on this). I can not understand how to describe the new system X .Thanks. – Robin Mar 29 '16 at 23:31
• @Robin: I've added a diagram. Perhaps you can fill out the transition matrix. I would order the states as follows: $A_0, B_0, A_1, B_1, A_2, B_2, A_3, B_3, \ldots$, since you can't very well put all the $A_i$ before all of the $B_i$. – Brian Tung Mar 29 '16 at 23:57