• $X_0,X_1,...$ irreducible, recurrent Markov chain with transition matrix $P$
  • Starting state $X_0=x$
  • $g(m)=P\{X_m=y\}$ for some fixed state $y$

I know that the renewal process is $g(m)=b(m)+\sum\limits_{k=1}^m P(Y_1=k)\cdot g(m-k)$, where $Y_1$ is the size of any jump from a state to state and $b(m)$ is another some sequence.

I have a trouble linking Discrete Markov chain and renewal theory. I know that every state change in markov is renewal theory but it seems hard to answer the following questions.

  1. how to construct a renew equation for g(m)?
  2. $lim_{m\rightarrow\infty} P\{X_m=y\}$ - I suppose this can be answered easily by Key Renewal Theorem if I figure out what $g(m)$ is though
  3. Do we get the same renewal equation if $X_0,X_1,...$ is a regenerative process, instead of MC? If not, what is the renewal equation?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.