# Define Markov chain and rewrite to recursively solve

Customers arrive at a server with rate $\lambda$ and are served at rate $\mu$. The server breaks down with rate $\gamma$, which causes all customers to leave. New customers can only arrive once the server has been repaired, which happens at rate $\delta$. Model as a MC give balance equations, and rewrite those so that they can be recursively solved (i.e. express for instance $\pi_k$ in $\pi_{k-n}$ ($n=1,\dots k$), $k=1,2,\dots$. You don't have to solve the equations.

My approach. Let $X(t)$ be the number of customers in the system and let the state space be $A=\{d, 0, 1, \dots\}$ where $d$ denotes a state in which the server is down.

The transition rates are then: $q_{d,0}=\delta$, $q_{i,d}=\gamma$, $q_{i,i+1}=\lambda$, $q_{i,i-1}=\mu$.

Then \begin{align*}\delta \pi_d &= \gamma \quad[\text{is this correct or should it be } \delta \pi_d = \gamma \pi_i, \ i =0,1,\dots?]\\ \lambda \pi_0 &= \mu \pi_1 + \delta \pi_d\\ (\lambda + \mu)\pi_i &= \lambda \pi_{i-1} + \mu \pi_{i+1}, \quad i \geq 2\\ \sum_i \pi_i&=1\end{align*}

Is my definition of the Markov chain correct and are the balance equations correct (specifically the first equation)? Also, how can I then rewrite the system so that it can be recursively solved like the question asks?

• Is there some context to determine what these variables actually mean? This problem is annoyingly ambiguous. What does it mean to repair a server at rate $\delta$? What does it mean to breakdown at rate $\gamma$ ? Probability of breakdown per customer? Amount of breakdowns per unit time? This problem seems to be a guessing game. – DanielV Feb 8 '15 at 10:01
• @DanielV I agree with you but I know from experience that when my book writes it like this it means '$\delta$ is the rate at which servers are repaired' and '$\gamma$ is the rate at which servers break down'. – aawerawer Feb 8 '15 at 10:03
• That would be the context that the rest of us have to guess. All you did was repeat what the problem wrote, not add clarity. – DanielV Feb 8 '15 at 10:04
• @DanielV Well since my book always uses this way of expression I assumed it to be standard. Anyway, if it was unclear to you I apologize for the inconvenience. – aawerawer Feb 8 '15 at 10:05
• The statement of the exercise is not ambiguous since "rate" has a precise meaning in the context of the theory of Markov processes. But, more annoyingly, the first part of the exercise is to "Model as a MC" and yet we do not see the OP's model. Defining a Markov chain usually involves specifying some transition rates, where are these? – Did Feb 8 '15 at 10:09