# Continuous-time Markov chains: system with components in parallel and enough repairmen. How to calculate availability?

I am trying to solve a continuous-time Markov chain exercise. But I have been stuck for a few days and I don't know how to continue or even If what I have done is correct.

Consider a system having five identical independent components.

Each component works for an exponential time, measured in days, and parameter 4.

When a component breaks down, it is replaced immediately as sufficient technicians are available to address any failure at any time. The duration of a repair is also exponential and on average lasts half a day.

Determines, in the long run, how long the system is out of service because it has all its components damaged.

I consider state n as the state where exactly n components are under repair. So I built the transition matrix Q as below:

$$Q=\begin{pmatrix} -20 & 20 & 0 & 0 & 0 & 0\\ 2 & -18 & 16 & 0 & 0 & 0\\ 2 & 2 & -16 & 12 & 0 & 0\\ 2 & 2 & 2 & -14 & 8 & 0\\ 2 & 2 & 2 & 2 & -12 & 4\\ 2 & 2 & 2 & 2 & 2 & -10\\ \end{pmatrix}$$

I'm not totally sure that the half under the diagonal is totally correct.

If it is, any hint how to calculate the system availability in the long run?

• First of all, I would say your Q should be: $$Q=\begin{pmatrix} -20 & 20 & 0 & 0 & 0 & 0\\ 2 & -18 & 16 & 0 & 0 & 0\\ 0 & 4 & -16 & 12 & 0 & 0\\ 0 & 0 & 6 & -14 & 8 & 0\\ 0 & 0 & 0 & 8 & -12 & 4\\ 0 & 0 & 0 & 0 & 10 & -10\\ \end{pmatrix}$$ Commented May 13, 2022 at 0:42

Let's first assume $$j$$ components work and $$i$$ components are being repaired.

The probability that a broken component is fixed before a working component breaks down is $$\mathbb{P}\Big(\min\{T_1,...,T_j\}>\min\{S_1,...,S_i\}\Big)$$ Here $$T_1,...,T_j\sim \text{Exponential}(4)$$ represent the times each of the $$j$$ components function properly while $$S_1,...,S_i\sim \text{Exponential}(2)$$ denote the repair times for each of the i broken components.

To calculate the aforementioned probability, set $$T=\min\{T_1,...,T_j\}$$ and $$S=\min\{S_1,...,S_i\}$$. It's easy to show that $$T\sim \text{Exponential}(4j)$$ and $$S\sim \text{Exponential}(2i)$$. Since $$S,T$$ are independent, the joint distribution of $$(S,T)$$ factors: $$f_{S,T}(x,y)=f_{S}(x)f_{T}(y)=8ij\exp\big\{-(2ix+4jy)\big\}$$ Then $$\mathbb{P}(T>S)=\int_0^{\infty}\int_x^{\infty}f_{S,T}(x,y)\mathrm{d}y\mathrm{d}x=\frac{i}{i+2j}$$

If $$X_n$$ is the number of components that function properly at time $$n$$, we see that conditional distribution $$X_{n+1}|X_n=j$$ for $$1\leq j \leq 4$$ is supported on $$\{j-1,j+1\}$$ and has pdf $$\mathbb{P}\left(X_{n+1}=j+1|X_{n}=j\right)=\frac{5-j}{5+j}$$ $$\mathbb{P}\left(X_{n+1}=j-1|X_n=j \right)=\frac{2j}{5+j}$$ Clearly $$\mathbb{P}\left(X_{n+1}=1|X_n=0\right)=\mathbb{P}\left(X_{n+1}=4|X_n=5\right)=1$$. Since the exponential distribution is memoryless, $$\{X_n\}_{n\geq 0}$$ is a Markov chain with state transition matrix $$P=\begin{pmatrix}0&1&0&0&0&0\\ \frac{1}{3}&0&\frac{2}{3}&0&0&0\\ 0&\frac{4}{7}&0&\frac{3}{7}&0&0\\ 0&0&\frac{3}{4}&0&\frac{1}{4}&0\\ 0&0&0&\frac{8}{9}&0&\frac{1}{9}\\ 0&0&0&0&1&0\end{pmatrix}$$ The unique stationary distribution of this Markov chain is $$\vec{\pi}=\begin{pmatrix}\frac{8}{81}\\ \frac{8}{27}\\ \frac{28}{81}\\ \frac{16}{81}\\ \frac{1}{18}\\ \frac{1}{162}\end{pmatrix}$$ The expected dwelling time in state $$0$$ is precisely equal to the expected value of $$\min\{S_1,...,S_5\}\sim \text{Exponential}(10)$$ which is $$\frac{1}{10}$$. So from this we see limiting expected holding time in state $$0$$ is $$\frac{1}{10}\pi_0=\frac{4}{405}$$.

• Given that there are 6 different states {0, 1, 2, 3, 4, 5, 6}, Why is P a 5x5 matrix, instead of 6x6? Commented May 15, 2022 at 1:05
• You are right. There are six states in the jump chain, and they are $\{0,1,2,3,4,5\}$. I will fix this momentarily. Thank you for pointing this out.
– user801306
Commented May 15, 2022 at 1:07
• If you could please add a small hint on how to derive the first three probabilities before the transition matrix I'll accept the answer and assign the bounty Commented May 15, 2022 at 15:09