Markov Chain mean hitting time Let the transition matrix be 
$P = \begin{pmatrix} 0 & 1/3 & 2/3 & 0 & 0 \\
0 & 0 & 0 & 1/2 & 1/2 \\
0 & 0 & 0& 1/4 & 3/4 \\
1 & 0 & 0 & 0 & 0 \\
1/2 & 0 & 0 & 0 & 1/2 
\end{pmatrix}$ with states $\{0,1,2,3,4\}$
The problems are:
(a) Suppose $X_0 = 0$. What is the expected number of steps until the chain is in state 3 ?
(b) Suppose $X_0 = 0$. What is the probability that the chain will enter state 4 before it enters state 2?
For part (a) I figure out a solution, could you check the correctness for me?
Let $T_3:=$ the first time the chain visit state 3, then   
$ \begin{align}
E[T_3 | X_0 = 0] & =  P(X_1 = 1| X_0 = 0) E[T_3| X_1 = 1] + P(X_1=2 | X_0=0) E[T_3|X_1 = 2]\\
& = \frac{1}{3}(1 + E[T_3|X_0 = 1]) + \frac{2}{3}(1 + E[T_3|X_0=2]) \\
& = 1 + \frac{1}{3}E[T_3|X_0 = 1] + \frac{2}{3}E[T_3|X_0 = 2] \\
& = 1+ \frac{1}{3}[\frac{1}{2}(1 + E[T_3|X_0 = 3] + \frac{1}{2}(1 + E[T_3|X_0 = 4])] + \frac{2}{3}[\frac{1}{4}(1+E[T_3|X_0=3]) + \frac{3}{4}(1+E[T_3|X_0=4])] \\
& = \frac{7}{3} + \frac{2}{3}E[T_3|X_0=4]
\end{align}$
Since $E[T_3|X_0 = 4] = \frac{1}{2}E[T_3|X_1 =0] + \frac{1}{2}E[T_3|X_1 =4] = 1 + \frac{1}{2}E[T_3|X_0=0] +\frac{1}{2}E[T_3|X_0 =4]$
Equating the two equality we have $E[T_3|X_0 =0] = 11$
Thank you very much!
 A: You have to solve the following recursive relations.  The current state is 0 Let $h(k)$ be the expected number of steps until your reach the (absorbing) state $3$ when you are in state $k$, for $k=0,1,2,4$. So we have that $$h(3)=0$$ because when you are already in $3$ you need zero steps to reach $3$. Then for $k=4$ $$h(4)=1+0.5h(0)+0.5h(4)$$ because when you are in state 4 you will do one step $(+1)$ and you will reach with probability $0.5$ again state $0$ and with probability $0.5$ state $4$. And you start over (to count the expected number of steps) from the new state, therefore $0.5h(4)$ and $0.5h(0)$. Similarly you can determine $h(2)$ and $h(1)$ and $h(0)$ and solve the system of equations to determine $h(0)$ which is the expected value of steps to reach state $3$ from the initial state which is $0$. More precisely:
$$h(0)=1+\frac{1}{3}h(1)+\frac{2}{3}h(2)$$ and $$h(1)=1+0.5h(3)+0.5h(4)$$ and $$h(2)=1+0.25h(3)+0.75h(4)$$which when you solve the system of equations gives $h(0) = 10$.  (if I have not made any mistake). (Please doublecheck the calculations because I am very prone to mistakes in the calculations). So the expected number of steps is $\boxed{10}$.
Complete the state diagram

Part B)

The required probability $$= \boxed {0.2}$$
