# Finding the generator matrix $Q$ of a markov chain.

The question:

A facility has three machines and three mechanics. Machines break down at a rate of $$1$$ per $$24$$ hours. Breakdown times are exponentially distributed. The time it takes a mechanic to fix a machine is exponentially distributed with mean $$6$$ hours. Only one mechanic can work on a failed machine at any given time. Let $$X_t$$ be the number of machines working at time $$t$$. Find the long-term probability that all machines are working.

My issue is with finding the rate matrix $$Q$$. Is it as simple as I think? Could I just say it is

$$Q= \begin{pmatrix} -\frac16&\frac16&0&0\\ \frac1{24}&-\frac5{24}&\frac16&0\\ 0&\frac1{24}&-\frac{5}{24}&\frac16\\ 0&0&\frac1{24}&-\frac{1}{24} \end{pmatrix}\ \ ?$$

My only problem is figuring $$Q$$ out, please do not provide an answer with the long-term probability. I'd prefer to do that myself. I just want to be sure that this $$Q$$ is correct, but I've a feeling it isn't considering I never used the fact that they are exponentially distributed. If this isn't the correct approach, would it be better to find the transition probability matrix function $$P(t)$$ first and then calculate $$Q$$ from there? If so, any tips on how I'd find this? Thanks.

Remember the minimum of $$n$$ exponentially distributed random variables is in fact another exponentially distributed random variable, with rate the sum of the other rates, i.e. $$X_n\sim Exp(\lambda_n) \\ \implies \min\big( X_1,\ldots,X_n \big) \sim Exp\big( \sum_{i=1}^n \lambda_n \big).$$
• That's what I get: $$Q=\pmatrix{-\frac{1}{2}&\frac{1}{2}&0&0\\ \frac{1}{24}&-\frac{3}{8}&\frac{1}{3}&0\\ 0&\frac{1}{12}&-\frac{1}{4}&\frac{1}{6}\\ 0&0&\frac{1}{8}&-\frac{1}{8}}\ .$$ I calculated the third row in detail from first principles. The remaining entries are educated guesses. Apr 19, 2020 at 1:08
• Absolutely! State 0: 3 workmen so jump to state 1 with rate $q_{12}= 3/6=1/2$. Since there is no other state reachable from state 0, the rate of staying in state 0 is $-1/2$, so the row sums to zero. State 1: 2 workmen so jump to state 2 with rate $2/6=1/3$, and one machine that can break at rate $1/24$ to return to state 0 and the diagonal term makes the row sum to zero, ie $-3/8$. State 2: one workman with 2 machines which can break; $\left( 0,1/12,-1/4,1/6\right)$. State 3: all three machines work for a rate of $3/24=1/8$ to drop down, diagonal as before. So you are correct Ionza and John! Apr 21, 2020 at 1:13