2
$\begingroup$

Certain machine has three possible states: $0=working,\,1=broken\,and\, awaiting\,repair,\,2=broken\,and\,being\,repaired$. The permanence times (in minutes) in each state have independent geometric distributions with means $\mu_0=5$, $\mu_1=3$ and $\mu_2=2$, resp. At the end of the permanence time in a state, the next state to be chosen is given by the following transition matrix: $$ \begin{bmatrix} 0 & 3/4 & 1/4 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \\ \end{bmatrix} $$ For $X_n \in \{0,1,2\}$ defining the machine state at the time $n$ ("working", "broken and awating repair" and "being repaired")

a) Show that $\{X_n\}_{n\ge0}$ is a Markov chain.
b) Determine the transition-probabilities matrix of $\{X_n\}_{n\ge0}$.
c) Determine the stationary distribution of $\{X_n\}_{n\ge0}$.
d) Is $\{X_n\}_{n\ge0}$ reversible?

I'm completely stuck since I've problems with formal definitions (a and d) and I thought that the given matrix was already the transition-probability matrix (b and c).

Any help is greatly appreciated!

$\endgroup$

1 Answer 1

2
$\begingroup$

You are given the transition matrix for what happens on exiting permanence states.   That is not for what happens at the end of any minute; only what happens when a state change does finally occur.

You are also informed that the permanence time (a count of minutes) is itself a geometric random variable with given means.   That tells you the probability that the system will stay or exit permanence at the end of any given minute (for each state).

Your task is to combine these facts to determine the probabilities for staying in the current state, or transitioning to either one of the others, for any given state at the start of the minute.   That is your transition-probabilities matrix.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .