# Computer failure with Markov chains and n-step transition matrix

Hi I am struggling with a Markov Chain question:

A computer network has two servers, only one of which is in operation at any given time. A server may break down on any given day with probability p. There is a single repairman that requires two days to restore the server to normal. The repairman can only work on one server at a time. A Markov chain is formed by taking as states the pairs (x, y), where x is the number of servers working at the end of a day and y is 1 if a day’s work has been expended on a machine not yet repaired and 0 otherwise.

I understand that: The four possible states are {(2,0),(1,0),(1,1),(0,1)} and The Transition matrix is {{1-p,p,0,0},{0,0,1-p,p},{1-p,p,0,0},{0,1,0,0}}

Show that the probability that the computer network is running is 1/(1+p^2)

• The probability that it is running at what time? Nov 10 '14 at 1:38
• It doesn't specify what time, just that the computer network is running!
– user191211
Nov 10 '14 at 2:55
• If it helps the second question is: Suppose that a second repairman is employed, so that both servers can be repaired simultaneously. Show that the probabilty that the computer network is available is now: (1+p)/(1+p+p^2) I am also stuck on this
– user191211
Nov 10 '14 at 3:03

They are asking for the probability in the stationary distribution. To find this, let $M$ be the transition matrix and let $I$ be the identity matrx ($1$'s on the diagonal, $0$'s elsewhere). Solve the matrix equation $$(M^T-I)v=0$$ where $M^T$ is the transpose of $M$. This is more commonly shown as $$v(M-I)=0$$ In the first equation $v$ is a column vector, and in the second $v$ is a row vector.
After finding the solutions (there will be infinitely many, parametrized by one variable) normalize the vector so that the sum of the components is $1$. Then the sum of the first three components (or alternatively $1$ minus the fourth component) is the result you are looking for.