# Mean return time Markov Chain

A Markov chain has states $$0,1,2$$ with transition probabilities $$P=\begin{pmatrix} 0.8 & 0.1 & 0.1 \\ 0.3 & 0.5 & 0.2 \\ 0.2 & 0.4 & 0.4 \end{pmatrix}.$$

I struggle to calculate the mean return time to state 1 given that we start at state 1.

Define $$T=\min{\{n\geq1 :X_n=1\}}$$ and $$g_i=E(T|X_0=i)$$. I seek $$g_1$$. By total probability and Markov property $$g_i=\sum_{j\in {1,2,3}}E(T|X_0=i,X_1=j)P(X_0=i,X_1=j))) =\sum_{j\in {1,2,3}} E(T+1|X_0=j)P(X_0=i,X_1=j))=\sum_{j\in {1,2,3}}g_ip_{ij}+1$$ So I get the system of equations $$\begin{cases} g_0 = 0.8g_0+0.1g_1+0.1g_2+1 \\ g_1 = 0.3g_0+0.5g_1+0.2g_2+1 \\ g_2 = 0.2g_0+0.4g_1+0.4g_2+1 \\ \end{cases}$$ which has no solution. What is wrong with my equations?

For all states $j \ne 1$, if you enter state $j$, you have on average $g_j$ more steps before you get to state $1$. However, if you enter state $1$, you don't have on average $g_1$ more steps before you get to state $1$: if you enter state $1$, you're done!

In other words, $E(T \mid X_0 = i, X_1 = 1)$ should just simplify to $1$, unlike every other instance of $E(T \mid X_0 = i, X_1 = j)$, because if we know that $X_1 = 1$, we know that $T=1$.

So your system of equations should instead be: \begin{cases} g_0=0.8g_0+0.1\color{red}{\cdot 0}+0.1g_2+1 = 0.8g_0 + 0.1g_2 + 1\\ g_1=0.3g_0+0.5\color{red}{\cdot 0}+0.2g_2+1 = 0.3g_0 + 0.2g_2 + 1\\ g_2=0.2g_0+0.4\color{red}{\cdot 0}+0.4g_2+1 = 0.2g_0 + 0.4g_2 + 1\\ \end{cases}

• How does the 1 become a 0? Commented Mar 4, 2018 at 20:28
• It gets multiplied by the transition probability and becomes part of the $+1$ at the end. Commented Mar 4, 2018 at 20:46
• @MishaLavrov, let say we have found the systems' solution. What is a unit of mean return time here?
– Nick
Commented Jan 7, 2020 at 11:49
• The units are in number of steps. (So, if you are in state $1$, it takes $g_1$ steps in expectation to return to state $1$ again.) Commented Jan 7, 2020 at 15:18
• It may help someone who couldn't understand the expressions at first like me, if we rewrite the expression of $g_0$ into $g_0=0.8(g_0+1)+0.1\cdot 1+0.1(g_2+1)= 0.8g_0 + 0.1g_2 + 1$. Similar rewriting for the expressions of $g_1, g_2$. Commented Jul 20, 2023 at 4:05