How can I find the average output of my Markov Chain based random number generator? I have a Markov Chain based random number generator which I've modeled here. It works by going along 5 points starting at the 1st point, each has a 25% chance to forward, a 25% chance to go backward, and a 50% chance to go the adding point connected to it. The adding points add 1 to the final number and always go back to the point they're connected to. The 1st point can't go backward so instead it would go back to itself. If the generator goes forward from the 5th point it goes into the final absorbing point which gives the final generated number.
I want to figure out on average what number would be given as the output, aka how many times one would expect the generator to go to the adding points. My thinking on how to figure this out is to find the probability of any given state in this chain and from there work out how many times one would expect the adding points to be visited. However, I have very basic knowledge of Markov Chains and I'm not sure how I would do this or if this is even the right approach.
 A: Let $Y$ be the final outcome. The key to solving this kind of problem is to consider the conditional expectation from all the possible starting points in the chain. This means, that we need to consider $\mathbb{E}[Y \: | \: X_0 = x_0]$ for all $x_0 \in \{A,a,B,b,C,c,D,d,E,e,F\}$ with $F$ being the final state.
Clearly $\mathbb{E}[Y \: | \: X_0=F] = 0$. Also it should be clear that
$$\mathbb{E}[Y\: | X_0=a] = 1 + \mathbb{E}[Y\: | X_0 = A],$$
and the same holds for the other pairs $(B,b),\dots,(E,e)$. To compute $\mathbb{E}[Y\: | \:X_0 = A]$ we can consider the following
\begin{align*}
\mathbb{E}[Y \: | \: X_0 = A] &= 0.25\mathbb{E}[Y \: | \: X_0 = A] +0.5\mathbb{E}[Y \: |X_0 =a] + 0.25\mathbb{E}[Y \: | \: X_0 = B] \\
&= 0.25\mathbb{E}[Y \: | X_0 = A] + 0.5 (1+\mathbb{E}[Y \: | X_0 =A) + 0.25\mathbb{E}[Y\: | \:X_0 =B]
\end{align*}
which is equivalent to
$$\mathbb{E}[Y \: | \: X_0 = A] - \mathbb{E}[Y\: | \: X_0 = B] = 2.$$
We notice that we get a linear equation. Repeating the same procedure for $B,C,D$ and $E$ we get another 4 linear equations, such that we have 5 equations in 5 variables. If i did the math correctly, then the system of equations becomes
$$\begin{pmatrix}0.25   & - 0.25 &    0  & 0     & 0     \\
                 -0.25 & 0.5   & -0.25 & 0     & 0    \\
                  0    & -0.25 & 0.5   & -0.25 & 0     \\
                  0    &   0   & -0.25 & 0.5   & -0.25 \\
                  0    &   0   &  0    & -0.25 &  0.5\end{pmatrix}
  \begin{pmatrix}  \mathbb{E}[Y \: | \: X=A] \\ 
                   \mathbb{E}[Y \: | \: X=B] \\ 
                   \mathbb{E}[Y \: | \: X=C] \\ 
                   \mathbb{E}[Y \: | \: X=D] \\ 
                   \mathbb{E}[Y \: | \: X=E]  \end{pmatrix}  
   = \begin{pmatrix}0.5 \\ 0.5 \\ 0.5 \\ 0.5 \\ 0.5 \end{pmatrix}$$
to which the solution is
$$\begin{pmatrix}  \mathbb{E}[Y \: | \: X=A] \\ 
                   \mathbb{E}[Y \: | \: X=B] \\ 
                   \mathbb{E}[Y \: | \: X=C] \\ 
                   \mathbb{E}[Y \: | \: X=D] \\ 
                   \mathbb{E}[Y \: | \: X=E]  \end{pmatrix}  
    =\begin{pmatrix} 30 \\ 28 \\ 24 \\ 18 \\ 10 \end{pmatrix}.$$
I assume, that your algorithm starts at $A$, so the answer will be $30$.
