0
$\begingroup$

Hi I am struggling with a Markov Chain question:

A computer network has two servers, only one of which is in operation at any given time. A server may break down on any given day with probability p. There is a single repairman that requires two days to restore the server to normal. The repairman can only work on one server at a time. A Markov chain is formed by taking as states the pairs (x, y), where x is the number of servers working at the end of a day and y is 1 if a day’s work has been expended on a machine not yet repaired and 0 otherwise.

I understand that: The four possible states are {(2,0),(1,0),(1,1),(0,1)} and The Transition matrix is {{1-p,p,0,0},{0,0,1-p,p},{1-p,p,0,0},{0,1,0,0}}

Show that the probability that the computer network is running is 1/(1+p^2)

I might be making this more difficult than it is, but please help!

$\endgroup$
3
  • $\begingroup$ The probability that it is running at what time? $\endgroup$ Nov 10 '14 at 1:38
  • $\begingroup$ It doesn't specify what time, just that the computer network is running! $\endgroup$
    – user191211
    Nov 10 '14 at 2:55
  • $\begingroup$ If it helps the second question is: Suppose that a second repairman is employed, so that both servers can be repaired simultaneously. Show that the probabilty that the computer network is available is now: (1+p)/(1+p+p^2) I am also stuck on this $\endgroup$
    – user191211
    Nov 10 '14 at 3:03
0
$\begingroup$

They are asking for the probability in the stationary distribution. To find this, let $M$ be the transition matrix and let $I$ be the identity matrx ($1$'s on the diagonal, $0$'s elsewhere). Solve the matrix equation $$(M^T-I)v=0$$ where $M^T$ is the transpose of $M$. This is more commonly shown as $$v(M-I)=0$$ In the first equation $v$ is a column vector, and in the second $v$ is a row vector.

After finding the solutions (there will be infinitely many, parametrized by one variable) normalize the vector so that the sum of the components is $1$. Then the sum of the first three components (or alternatively $1$ minus the fourth component) is the result you are looking for.

For the second problem mentioned in the comments you do the same thing, but the transition matrix will be different.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy