1
$\begingroup$

GIVEN:

  • $X_0,X_1,...$ irreducible, recurrent Markov chain with transition matrix $P$
  • Starting state $X_0=x$
  • $g(m)=P\{X_m=y\}$ for some fixed state $y$

I know that the renewal process is $g(m)=b(m)+\sum\limits_{k=1}^m P(Y_1=k)\cdot g(m-k)$, where $Y_1$ is the size of any jump from a state to state and $b(m)$ is another some sequence.

I have a trouble linking Discrete Markov chain and renewal theory. I know that every state change in markov is renewal theory but it seems hard to answer the following questions.

  1. how to construct a renew equation for g(m)?
  2. $lim_{m\rightarrow\infty} P\{X_m=y\}$ - I suppose this can be answered easily by Key Renewal Theorem if I figure out what $g(m)$ is though
  3. Do we get the same renewal equation if $X_0,X_1,...$ is a regenerative process, instead of MC? If not, what is the renewal equation?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.