# Markov chain - transition matrix - average return time

Given is a markov chain with transition matrix

$M=\begin{pmatrix} 0.5 & 0.5 & 0 & 0 & 0 & 0\\ 0.25 & 0.75 & 0 & 0 & 0 & 0\\ 0.25 & 0.25 & 0.25 & 0.25 & 0 & 0\\ 0.25 & 0 & 0.25 & 0.25 & 0 & 0.25\\ 0 & 0 & 0 & 0 & 0.5 & 0.5\\ 0 & 0 & 0 & 0 & 0.5 & 0.5 \end{pmatrix}$ with states $S=\left\{0,1,2,3,4,5\right\}$

Calculate the average return time to state $1$, e.g. $\mu_1=E(T_1 \mid X_0=1)$ where $T_1 = \inf\left\{n \geq 1: X_n=1\right\}$

I'm learning for an exam and here is another example task I tried. But I don't know if my thoughts and solution are correct?

So I don't really understand that weird notation at the bottom of the yellow marked text but luckily, it's mentioned that state $1$ is meant by it.

For this reason, I created following linear system:

I: $h(0) = 1+0.5h(0)+0.5h(1)\;\;\;\;$ by that I mean: If we are in state $0$, we go to state $0$ with a probability of $0.5$ and with probability of $0.5$, we go to state $1$

II: $h(1) = 0\;\;\;\;$ because that's the state where we end (in $1$)

We do not need further equations because every other value in row $1$ is zero anyway.

Solving the small system above: $h(0) = 1+0.5h(0)+0.5h(1) \Leftrightarrow h(0)=1+0.5h(0) \Leftrightarrow 0.5h(0)=1 \Leftrightarrow h(0)=2$

Now to get the average return time to state $1$, we do

$x=1+h(0) = 1+2=3$

Thus, the average return time to state $1$ is $3$.

## 1 Answer

That's not quite right, because there is a possibility to go from $1$ right back to itself. You should have $x=1+0.25h(0)$.