Expected Run before Absorption Consider a $4$ state Markov Chain that includes the row-stochastic
transition matrix
$$P = \left(\begin{array}{cccc} 1/4&1/4&1/4&1/4 \\ 1/2&0&0&1/2 \\
                                1/3&1/3&1/3&0 \\ 0&0&0&1 \end{array}
\right).$$
Assume that we start the chain in state $1$. I would like to find the expected
length of time intervals until we are absorbed (as in, before we reach state 4).
To me, this seems like a problem where we have a large number of potential
transitions, and I need some help on considering a clean way to calculate the
terms for the probability that the expected length is $\geq 3$. I see that for
the first term of this expectation, we have
$1 \times P_{1,4} = \frac{1}{4},$ and for the second term, we have
$$2 \times \sum_{i=1}^3 P_{1,i}P_{i,4}
= 2 \times (\frac{1}{4} \times \frac{1}{4} + \frac{1}{4} \times \frac{1}{2}
+ \frac{1}{4} \times 0)
= 2 \times (\frac{3}{16}) = \frac{3}{8}.$$
However, I see how this computation becomes much more complicated given the
length of the chain. Is there potentially a more efficient way to calculate
the individual terms of this expectation?
 A: Considering this, 
assume $$Q = \left(\begin{array}{cccc} 1/4&1/4&1/4 \\ 1/2&0&0 \\
                                1/3&1/3&1/3 \end{array}
\right)$$
Then the expected number of transitions from state $i$ to $j$ is 
$$N=(I-Q)^{-1}=\left(\begin{array}{cccc} 2.2857 &0.8571&0.8571 \\ 1.1429 &  1.4286 & 0.4286\\
                                1.7143 & 1.1429 & 2.1429\end{array}
\right)$$
The $i$th element of the vector $\mathbf{t}=N\mathbf{1}$ gives the expected number of transitions until absorption when beginning from state $i$.
For this case, $$\mathbf{t}=\left(\begin{array}{cccc} 2.2857 &0.8571&0.8571 \\ 1.1429 &  1.4286 & 0.4286\\
                                1.7143 & 1.1429 & 2.1429\end{array}
\right)\begin{pmatrix}
1\\ 
1\\ 
1
\end{pmatrix}=\begin{pmatrix}
4\\ 
3\\ 
5
\end{pmatrix}$$
So the answer to your question is $\color{red}{4}$.
A: You don’t really have to compute each path length probability to solve this. Let $v_i$ be the expected number of steps to first reach state 4, the absorbing state, having started in state $i\ne4$. This number includes the first step plus the expected number of steps until absorption starting from whatever state was entered. We thus get the system of linear equations $$v_i=1+\sum_{i\ne4}P_{ij}v_j\tag{1}$$ for $i\ne4$. If we set $v_4=0$ and define $\mathbf r$ as $r_i=1$ for $i\ne4$ and $r_4=0$, this can be expressed in matrix form as $\mathbf v=\mathbf r+P\mathbf v$ or, equivalently, $$(I-P)\mathbf v=\mathbf r.\tag{2}$$ Solving this system yields the expected absorption times. If we omit the all-zero last row and the irrelevant $v_4$ column, this becomes $(I-Q)\mathbf v=\mathbf 1$, with $Q$ defined as in the wikipedia page cited in msm’s answer, and so $\mathbf v=(I-Q)^{-1}\mathbf 1$. (I’ll leave a proof that $I-Q$ is nonsingular for another time.)  
This system can, of course be solved via standard row-reduction: $$\left[\begin{array}{ccc|c}\frac34&-\frac14&-\frac14&1\\-\frac12&1&0&1\\-\frac13&\frac13&\frac23&1\end{array}\right]$$ reduces to $$\left[\begin{array}{ccc|c}1&0&0&4\\0&1&0&3\\0&0&1&5\end{array}\right],$$ i.e., $v_1=4$, $v_2=3$ and $v_3=5$.

This can instead be solved via an infinite sum. For $k\ge0$ let $A_k$ be the event “not absorbed after $k$ steps” with $I_k$ the corresponding indicator variable. Then, with a big handwave regarding convergence, the expected time until absorption is $$E\left[\sum_{k=0}^\infty I_k\right] = \sum_{k=0}^\infty E[I_k] = \sum_{k=0}^\infty \operatorname{Pr}(A_k).$$ Recall that $P^k$ gives the $k$-step transition probabilities, so $P^k\mathbf r$, with $\mathbf r$ defined as above, is a vector that gives us $\operatorname{Pr}(A_k)$ for all starting states. Thus $\sum_{k=0}^\infty P^k\mathbf r$ is a vector that consists of all of the expected absorption times. We can again discard the last row and column of $P$ (why?), leaving $\sum_{k=0}^\infty Q^k\mathbf 1=(I-Q)^{-1}\mathbf 1$ as before.
