Starting from state 0, the mean number of visits of state 2 before coming back to state 0 is? The transition probability matrix is$$
        \mathrm P= \begin{pmatrix}
        0 & 2/3 & 1/3 \\
        1/2 & 0 & 1/2 \\
        1/2 & 1/2 & 0 \\
        \end{pmatrix}
$$
Starting from state 0, the mean number of visits of state 2 before coming back to state 0 is


*

*5/9

*6/9 

*7/9

*8/9

*None of the above is correct.


If my step is from 0->1->2->0, the probability = $1/6$, but I don't know the total step from state 0 toward itself, and also what is the meaning of "the mean number of visits"?
 A: Let $m_i$ be the mean number of visits to state $2$ before returning to state $0$, starting from state $i$. We want to determine $m_{0}$. By using a one-step analysis, these $m_{i}$ satisfy
\begin{align}
m_{0} &= \frac{2}{3} m_{1} + \frac{1}{3} m_{2}, \\
m_{1} &= \frac{1}{2} \cdot 0 + \frac{1}{2} m_{2}, \\
m_{2} &= 1 + \frac{1}{2} \cdot 0 + \frac{1}{2} m_{1}.
\end{align}
The interpretation is as follows. Since this is a Markov chain: if I start in state $0$, with probability $\frac{2}{3}$ I transition to state $1$ and can start the process from there and with probability $\frac{1}{3}$ I transition to state $2$ and can start the process from there. The $\frac{1}{2} \cdot 0$ terms indicate that with probability $\frac{1}{2}$ we have returned to state $0$ and the excursion ends. The $1$ term in the $m_2$ expression counts that the process has spent 1 time unit in state $2$. 
You can probably solve this system of equations.
A: The "mean number of visits of state $2$" mean how many times do you visit state $2$ on average.Or also the expected number of visit of state $2$. So the mean number of visit is given by the sum, over all runs going from $0$ to $0$, of the probability of the run times the number of times it visit $2$.
So the example you gave ($0\to1\to2\to 0$) gives you that the average number of visit is greater than $1/6*1=1/6$ (it is greater since you only considered one run, you should consider all runs to get the right value.
Denoting $E(n)$ the expected number of visit of state 2 from state $n$, you can right the 3 equation defining $E(0),E(1)$ and $E(2)$ solving this equation will give you the value you look for (it is $E(0)$).
Below are the 3 equations, please do not look at it if you didn't understand something. Ask if something is not clear I'll try to help/be clearer.

 $$E(0) = 2/3 E(1)+1/3E(2)$$

$ $

 $$E(1) = 1/2 E(2)$$

$ $

 $$E(2) = 1 + 1/2 E(1)$$

