I am a bit confused with the following:

Suppose that we have a Markov Chain and a state $i$ that is transient, that is $f_i<1$. Since $i$ is transient, we cannot define the average time of recurrence of $i$ (it is only defined for recurrent states) but we define the average number of visits of $i$, given that the Chain begins its movement from $i$. How can we compute that average without using geometric transformations? My guess was that the sum $\displaystyle{\sum_{n=1}^\infty nf_{ii}^{(n)}}$ would be the desired average but I have nothing like that in my notes. Any help?

  • $\begingroup$ Not sure if this will help, but you could replace the non-transient states with some number of absorbing states. The fundamental matrix of the modified chain gives you the expected number of visits before absorption. $\endgroup$ – amd Jun 20 '18 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.