Probability Markov Chain Distribution The printing press in a newspaper has the following pattern of break-downs. If it is working today, tomorrow it has a 90% chance of working and a 10% chance of breaking down. If the press is broken today, it has a 60% chance of working tomorrow and a 40% chance of being broken again. 
Question: If the press is working today, what are the chances that it is working in 2 days' time? 
I spent the last hour trying to figure this out and I give up. At first, I tried using a Markov Chain table that plots the value of current and future states of the press working and breaking down. 
The problem is, when I only work with a 2 by 2 table, it is very simple.. but once I need to find the probability of a 3rd day, I simply couldn't settle on any definitive method. 
In the end I just tried using the most brute and forced way I could think of.. 
Day 1 = 100%
Day 2 = 90% 
Day 3 = 90% * 1/2
So in total I have (1 + 9/10 + 9/20)/3 
Basically, the first day I am told the press is working so its 100%. And since there is a 90% chance of the press working the following day if it is working today, Day 2 then simply has a 90% chance of the press working. 
Day 3 is when it gets tricky because in order for the press to be working on day 3, I thought it would need to pass the 90% test twice so I divided 90% by 2 to get 45%. 
Lastly, I just averaged the values ( 1 + 9/10 + 9/20)/3.
However, I also thought that since day 1 is given, there is no need to compute it and I only need to calculate day 2 and day 3 which is, (9/10 + 9/20)/2... But I don't know anymore. Please help someone..anyone.. 
 A: Hint:
The diagram below represents the state transitions in the given system.

Notice that there are two cases that you need to consider:


*

*Press working today $\longrightarrow$ press working tomorrow $\longrightarrow$ press working day after tomorrow

*Press working today $\longrightarrow$ press broken tomorrow $\longrightarrow$ press working day after tomorrow


The probability of the first case is given by $$\mathbb P\left(\mathrm W_0\to\mathrm W_1\to\mathrm W_2\right) = \mathbb P\left(\mathrm W_0\to\mathrm W_1\right)\cdot\mathbb P\left(\mathrm W_1\to\mathrm W_2\right).$$
Similarly, for the second case, we have $$\mathbb P\left(\mathrm W_0\to\mathrm B_1\to\mathrm W_2\right) = \mathbb P\left(\mathrm W_0\to\mathrm B_1\right)\cdot\mathbb P\left(\mathrm B_1\to\mathrm W_2\right).$$
[$\mathrm X_i\to\mathrm Y_j$ represents transitioning from state $\mathrm X$ on the $i^{th}$ day to state $\mathrm Y$ on the $j^{th}$ day]
A: Let $X_n\in\{U,D\}$ for $n=0,1,2,\ldots$ denote whether the machine is up or down at time $n$. Then $\{X_n:n=0,1,2,\ldots\}$ is a Markov chain with transition matrix
$$
P = \begin{pmatrix} \frac9{10}&\frac1{10}\\\frac35&\frac25\end{pmatrix}.
$$
The probability in question is:
\begin{align}
\mathbb P(X_2 = U \mid X_0=U) &= \mathbb P(X_2=U\mid X_1=D)\mathbb P(X_1=D\mid X_0=U) + \mathbb P(X_2=U\mid X_1=U)\mathbb P(X_1=U\mid X_0=U)\\
&=\frac35\cdot\frac1{10} + \left(\frac9{10}\right)^2\\
&= \frac{87}{100}.
\end{align}
The $n$-step probabilities $\mathbb P(X_n=j\mid X_0=i)$ are given by
$$
P^n = \left(
\begin{array}{cc}
 \frac{1}{7} \left(\left(\frac{3}{10}\right)^n+6\right) & \frac{1}{7} \left(1-\left(\frac{3}{10}\right)^n\right) \\
 \frac{6}{7} \left(1-\left(\frac{3}{10}\right)^n\right) & \frac{1}{7}\cdot 10^{-n} \left(2\cdot 3^{n+1}+10^n\right) \\
\end{array}
\right).
$$
Because this is a finite-state, irreducible, aperiodic Markov chain, it has a unique stationary (and limiting) distribution $\pi_j := \lim_{n\to\infty}\mathbb P(X_n=j)$ independent of the starting state. We compute $\pi$ by taking the limit of $P^n$:
$$
\lim_{n\to\infty}P^n = \left(
\begin{array}{cc}
 \frac{6}{7} & \frac{1}{7} \\
 \frac{6}{7} & \frac{1}{7} \\
\end{array}
\right)
$$
So the long-run fraction of time the machine will be working is $6/7$.
