# Constructing the transition matrix for device failure

There are three machines. Let the probability that an operable machine fails on any given day be $0.1$, independently of the other machines. Only one machine can be repaired on the same day (so it is available for the next).

Machine $1$ feeds both machines $2$ and $3$. Thus, if machine $1$ fails, there is no production that day. If $1$ is down, it is always repaired first. If both $2$ and $3$ are down, and $1$ is up, then $2$ is repaired. Machines that are down remain down for the day. Machines that are up today may fail the next day, regardless if production is available or not. But a repaired machine is certain to operate the next day.

How would I construct the transition matrix for this problem? I have figured that there are $8$ state combinations:

$(1)$ - $1$ is down

$(2)$ - $2$ is down

$(3)$ - $3$ is down

$(4)$ - $1$,$2$ are down

$(5)$ - $1$,$3$ are down

$(6)$ - $2$,$3$ are down

$(7)$ - $1$,$2$,$3$ are down

$(8)$ - Everything works.

But I can't figure out how they would relate to each other (which states are transient and which ones are recurrent/absorbing). Any help would be greatly appreciated.

-
I edited the order of your states to make them more "ordered". – Patrick Da Silva Mar 13 '12 at 5:20

Clearly all states are recurrent ; there is always a non-zero probability of going from any of the states $1$-$7$ (the broken states) to state $8$ (the repaired state) in at most $3$ steps (repairing all the machines one by one without them breaking down) and at this step, we are in state $8$, and from state $1$ you can go to any other state in one step with non-zero probability. Therefore, starting somewhere, you might end up in any state four steps later with non-zero probability.

You are right, there are $\sum_{k=0}^3 \begin{pmatrix} 3 \\ k \end{pmatrix} = 2^3 = 8$ possible states (and you wrote them all down, that's even better). So you need to compute the probability of going from one state to another.

Say you are in a complicated state to compute ; suppose you are in state $(5)$. Machines $1$ and $3$ are broken, so in the next state, for sure we will have $1$ repaired and $3$ broken. But $2$ will be working with probability $0.9$, and will be broken with probability $0.1$. So from state $(5)$ you go to state $(3)$ with probability $0.9$, and to state $(6)$ with probability $0.1$.

You compute all the others this way. This gives you the matrix : $$\rho = \begin{bmatrix} 0 & 0.09 & 0.09 & 0 & 0 & 0.01 & 0 & 0.81 \\ 0.09 & 0 & 0.09 & 0 & 0.01 & 0 & 0 & 0.81 \\ 0.09 & 0.09 & 0 & 0.01 & 0 & 0 & 0 & 0.81 \\ 0 & 0.9 & 0 & 0 & 0 & 0.1 & 0 & 0 \\ 0 & 0 & 0.9 & 0 & 0 & 0.1 & 0 & 0 \\ 0 & 0 & 0.9 & 0 & 0.1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0.081 & 0.081 & 0.081 & 0.009 & 0.009 & 0.009 & 0.001 & 0.729 \\ \end{bmatrix}$$ where the $ij^{\text{th}}$ entry is the probability from going to state $i$ to state $j$, given the ordering above. (If you don't like my argument that all states are recurrent, compute the fourth power of this matrix with a computer and see that $\rho(i,i) > 0$ for $1 \le i \le 8$ for a more computational proof.)

Hope that helps,

-
This makes a lot of sense and just made everything very clear for me. Thanks! – Alistair Mar 13 '12 at 5:34
I added the matrix (it took me some time to write down all the numbers with TeX...) if you want to correct yourself. One quick way to compute it is to write down a 8 by 8 box and write the states in the form "(12)" or "(1)" or "(23)" beside each row/column, so that you can fill in many zeros by saying things such as "if we are in state (12), then 2 cannot be repaired at the next step, hence you can place zeros in line 4 and column 1, 3, 5, and 8." (I filled in all my zeros using such methods.) Afterwards there are only a few computations left that you can easily do by hand. – Patrick Da Silva Mar 13 '12 at 5:42
You can also add in zeros by saying "if we are in state (12), then 1 cannot be broken at the next step, so there are zeros in row 4 and column 1, 4, 5 and 7. With the preceding reasoning, you are left with only column 2 and 6 to determine in row 4. Not many computations left now. – Patrick Da Silva Mar 13 '12 at 5:47
This looks great, I have one question. For line 1 (state 1 being machine 1 is down), wouldn't column 8 be 0.9? Since getting from machine 1 is down to everything works involves only fixing Machine 1, isn't that just the probability 0.9 ? Same for line 2 and 3, column 8. – Alistair Mar 13 '12 at 5:55
Ok now I get it, we are just taking the probabilities of those that are left and not the one that is being repaired. – Alistair Mar 13 '12 at 6:07