# Show that it is a Markov chain, determine the transition-probability matrix and reversibility

Certain machine has three possible states: $0=working,\,1=broken\,and\, awaiting\,repair,\,2=broken\,and\,being\,repaired$. The permanence times (in minutes) in each state have independent geometric distributions with means $\mu_0=5$, $\mu_1=3$ and $\mu_2=2$, resp. At the end of the permanence time in a state, the next state to be chosen is given by the following transition matrix: $$\begin{bmatrix} 0 & 3/4 & 1/4 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \\ \end{bmatrix}$$ For $X_n \in \{0,1,2\}$ defining the machine state at the time $n$ ("working", "broken and awating repair" and "being repaired")

a) Show that $\{X_n\}_{n\ge0}$ is a Markov chain.
b) Determine the transition-probabilities matrix of $\{X_n\}_{n\ge0}$.
c) Determine the stationary distribution of $\{X_n\}_{n\ge0}$.
d) Is $\{X_n\}_{n\ge0}$ reversible?

I'm completely stuck since I've problems with formal definitions (a and d) and I thought that the given matrix was already the transition-probability matrix (b and c).

Any help is greatly appreciated!