-1
$\begingroup$

What is the formula to find average duration of state s in a Markov chain given a transition matrix?

I tried to recall the concept but could not find any references.

$\endgroup$

closed as off-topic by Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra Jan 25 at 15:02

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra
If this question can be reworded to fit the rules in the help center, please edit the question.

1
$\begingroup$

I understand that we are talking about a discrete time process. If the probability of staying in state $s$ from one period to another is $$P(X_n=s|X_{n-1}=s)=q,\quad \forall n\ge1,$$ then the number $N$ of consecutive repetitions of the state $s$ is a random variable with geometric distribution, that is $$N\sim \mathcal G(1-q),$$ since every transition can be seen as a dicotomic experiment where the "fail" consists in staying at state $s$ (with probability $1-q$) and "success" is changing to any other state (with probability $q$).

In fact, there are several alternative definitions of a geometric r.v., but here I consider it as the number of "fail" results before the first "success". In this case, we have $E(N)=\frac q{1-q}$.

Since the actual number of successive $s$ states would be $N+1$, if we consider the first time we reach that state, the expected number of $s$ states once the chain reaches it would be $$E(N+1)=1+\frac q{1-q}=\frac1{1-q},$$ counting the current time, or $$E(N)=\frac q{1-q},$$ if we count from next time on.

$\endgroup$
  • $\begingroup$ I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations... $\endgroup$ – zmicer Jan 25 at 6:48
  • 1
    $\begingroup$ Take $q=7/8$ and you'll get the result. $\endgroup$ – Alejandro Nasif Salum Jan 25 at 6:52
  • $\begingroup$ @ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks! $\endgroup$ – zmicer Jan 25 at 6:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.