# Linking Markov Chain with Renewal Process

GIVEN:

• $$X_0,X_1,...$$ irreducible, recurrent Markov chain with transition matrix $$P$$
• Starting state $$X_0=x$$
• $$g(m)=P\{X_m=y\}$$ for some fixed state $$y$$

I know that the renewal process is $$g(m)=b(m)+\sum\limits_{k=1}^m P(Y_1=k)\cdot g(m-k)$$, where $$Y_1$$ is the size of any jump from a state to state and $$b(m)$$ is another some sequence.

I have a trouble linking Discrete Markov chain and renewal theory. I know that every state change in markov is renewal theory but it seems hard to answer the following questions.

1. how to construct a renew equation for g(m)?
2. $$lim_{m\rightarrow\infty} P\{X_m=y\}$$ - I suppose this can be answered easily by Key Renewal Theorem if I figure out what $$g(m)$$ is though
3. Do we get the same renewal equation if $$X_0,X_1,...$$ is a regenerative process, instead of MC? If not, what is the renewal equation?