0
$\begingroup$

I am trying to solve a continuous-time Markov chain exercise. But I have been stuck for a few days and I don't know how to continue or even If what I have done is correct.

Consider a system having five identical independent components.

Each component works for an exponential time, measured in days, and parameter 4.

When a component breaks down, it is replaced immediately as sufficient technicians are available to address any failure at any time. The duration of a repair is also exponential and on average lasts half a day.

Determines, in the long run, how long the system is out of service because it has all its components damaged.

I consider state n as the state where exactly n components are under repair. So I built the transition matrix Q as below:

$$ Q=\begin{pmatrix} -20 & 20 & 0 & 0 & 0 & 0\\ 2 & -18 & 16 & 0 & 0 & 0\\ 2 & 2 & -16 & 12 & 0 & 0\\ 2 & 2 & 2 & -14 & 8 & 0\\ 2 & 2 & 2 & 2 & -12 & 4\\ 2 & 2 & 2 & 2 & 2 & -10\\ \end{pmatrix} $$

I'm not totally sure that the half under the diagonal is totally correct.

If it is, any hint how to calculate the system availability in the long run?

$\endgroup$
1
  • $\begingroup$ First of all, I would say your Q should be: $$ Q=\begin{pmatrix} -20 & 20 & 0 & 0 & 0 & 0\\ 2 & -18 & 16 & 0 & 0 & 0\\ 0 & 4 & -16 & 12 & 0 & 0\\ 0 & 0 & 6 & -14 & 8 & 0\\ 0 & 0 & 0 & 8 & -12 & 4\\ 0 & 0 & 0 & 0 & 10 & -10\\ \end{pmatrix} $$ $\endgroup$
    – LidRaRA
    May 13 at 0:42

1 Answer 1

1
+50
$\begingroup$

Let's first assume $j$ components work and $i$ components are being repaired.

The probability that a broken component is fixed before a working component breaks down is $$\mathbb{P}\Big(\min\{T_1,...,T_j\}>\min\{S_1,...,S_i\}\Big)$$ Here $T_1,...,T_j\sim \text{Exponential}(4)$ represent the times each of the $j$ components function properly while $S_1,...,S_i\sim \text{Exponential}(2)$ denote the repair times for each of the i broken components.

To calculate the aforementioned probability, set $T=\min\{T_1,...,T_j\}$ and $S=\min\{S_1,...,S_i\}$. It's easy to show that $T\sim \text{Exponential}(4j)$ and $S\sim \text{Exponential}(2i)$. Since $S,T$ are independent, the joint distribution of $(S,T)$ factors: $$f_{S,T}(x,y)=f_{S}(x)f_{T}(y)=8ij\exp\big\{-(2ix+4jy)\big\}$$ Then $$\mathbb{P}(T>S)=\int_0^{\infty}\int_x^{\infty}f_{S,T}(x,y)\mathrm{d}y\mathrm{d}x=\frac{i}{i+2j}$$

If $X_n$ is the number of components that function properly at time $n$, we see that conditional distribution $X_{n+1}|X_n=j$ for $1\leq j \leq 4$ is supported on $\{j-1,j+1\}$ and has pdf $$\mathbb{P}\left(X_{n+1}=j+1|X_{n}=j\right)=\frac{5-j}{5+j}$$ $$\mathbb{P}\left(X_{n+1}=j-1|X_n=j \right)=\frac{2j}{5+j}$$ Clearly $\mathbb{P}\left(X_{n+1}=1|X_n=0\right)=\mathbb{P}\left(X_{n+1}=4|X_n=5\right)=1$. Since the exponential distribution is memoryless, $\{X_n\}_{n\geq 0}$ is a Markov chain with state transition matrix $$P=\begin{pmatrix}0&1&0&0&0&0\\ \frac{1}{3}&0&\frac{2}{3}&0&0&0\\ 0&\frac{4}{7}&0&\frac{3}{7}&0&0\\ 0&0&\frac{3}{4}&0&\frac{1}{4}&0\\ 0&0&0&\frac{8}{9}&0&\frac{1}{9}\\ 0&0&0&0&1&0\end{pmatrix}$$ The unique stationary distribution of this Markov chain is $$\vec{\pi}=\begin{pmatrix}\frac{8}{81}\\ \frac{8}{27}\\ \frac{28}{81}\\ \frac{16}{81}\\ \frac{1}{18}\\ \frac{1}{162}\end{pmatrix}$$ The expected dwelling time in state $0$ is precisely equal to the expected value of $\min\{S_1,...,S_5\}\sim \text{Exponential}(10)$ which is $\frac{1}{10}$. So from this we see limiting expected holding time in state $0$ is $\frac{1}{10}\pi_0=\frac{4}{405}$.

$\endgroup$
3
  • $\begingroup$ Given that there are 6 different states {0, 1, 2, 3, 4, 5, 6}, Why is P a 5x5 matrix, instead of 6x6? $\endgroup$
    – Alfonso_MA
    May 15 at 1:05
  • $\begingroup$ You are right. There are six states in the jump chain, and they are $\{0,1,2,3,4,5\}$. I will fix this momentarily. Thank you for pointing this out. $\endgroup$
    – Matthew H.
    May 15 at 1:07
  • $\begingroup$ If you could please add a small hint on how to derive the first three probabilities before the transition matrix I'll accept the answer and assign the bounty $\endgroup$
    – Alfonso_MA
    May 15 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.