Layman perspective of mean time spent in transient state of a Markov chain.

Question

Let $X=\{X_n\}$ be a finite state Markov Chain with the state space $\{0,1,2,\ldots,N\}$ such that $0$ is the single absorbing state and all the rest states are transient. The following is the transition matrix.

$$ P = \left[\begin{matrix} 1 & 0 & \ldots & 0 \\ p_{10} & p_{11} & \ldots & p_{1N} \\ \vdots & \vdots & \ldots & \vdots \\p_{N0} & p_{N1} & \ldots & p_{NN} \end{matrix} \right] $$

Now, let $s_{ij}$; $1\leq{i,j}\leq{N}$ be the expected number of time periods that the Markov chain is in state $j$, given that it starts in state $i$. Then, $$ S:=(s_{ij})_{N\times{N}} =(I-P_T)^{-1} $$ where $$ P_T = \left[\begin{matrix} p_{11} & p_{12} & \ldots & p_{1N}\\ p_{21} & p_{22} & \ldots & p_{2N} \\ \vdots & \vdots & \ldots & \vdots \\p_{N1} & p_{N2} & \ldots & p_{NN} \end{matrix} \right] $$ If we let $f_{ij}$ be the probability that the Markov chain ever makes a transition into state $j$ given that it starts in state $i$, then $$ s_{ij} = (\delta_{ij}+s_{jj})f_{ij} + \delta_{ij}(1-f_{ij}) \tag{$*$} $$ where $\delta_{ij}$ is the Kronecker delta.

Now, my question is just the interpretation of the relation $(*)$. How do we explain relation $(*)$ in layman perspective? Or, how to explain relation $(*)$ to those people who do not know any mathematics?

NOTE: The above calculation can be found in the book "Introduction to Probability Models", by Ross.

Answer 1

How do we explain relation $(*)$ in layman perspective?

One must realize that $(*)$ is actually two series of identities written together. First, for every $i$, $$s_{ii}=1+f_{ii}s_{ii},$$ and second, for every $i\ne j$, $$s_{ij}=f_{ij}s_{jj}.$$ These identities are direct consequences of the (strong) Markov property of the Markov chain starting at $i$, considered at the first return time to $i$ (first case) and at the first hitting time of $j\ne i$ (second case).

Or, how to explain relation $(*)$ to those people who do not know any mathematics?

One cannot, this is a math question and the query is absurd.

Answer 2

I would rephrase $s_{ij} = (\delta_{ij}+s_{jj})f_{ij} + \delta_{ij}(1-f_{ij}) \tag{$*$}$ as

$s_{ij} = f_{ij} s_{jj} + \delta_{ij}\tag{$*$}$

which makes a layman's explanation a bit easier.

Also, remember that $$s_{jj} = 1 + \sum_{k=1}^{\infty}f_{jj}^k$$ which means that if $i=j$ then

$$\begin{align} f_{ij}s_{jj} + \delta{ij}&=f_{jj}*(1 + \sum_{k=1}^{\infty}f_{jj}^k) + 1 \\ &= 1 + f_{jj} + \sum_{k=1}^{\infty}f_{jj} * f_{jj}^{k} \\&= s_{jj} \end{align}$$

If you start at $j$, you get one point just for playing (that's your $\delta_{ij}$). Regardless of where you start, you get one point every time you transition to $j$. You might bust out first (hit an absorbing state), so you need to adjust for that probability (hence the $f_{ij}$). Then, once you're at state $j$, you already know how many times you expect to transition back ($s_{jj}$).

$(*)$ just says that the amount of time you expect to spend in state $j$ if you start at state $i$ is equal to the probability that you get there at all multiplied by the amount of time you expect to spend there once you've gotten there.