Kullback-Leibler divergence between two Markov Renewal Processes 
Consider the renewal process $P(x)$ is generated according to interrenewal distribution $p(x)$, and renewal process $Q(x)$ is generated according to interrenewal distribution $q(x)$. Calculate the Kullback–Leibler divergence between $P(x)$ and $Q(x)$.

Generally, the Kullback-Leibler divergence between two distributions $p(x)$ and $q(x)$ is
$ D_{\mathrm{KL}}(p\|q) = \int_{\chi} p \, \log \frac{p}{q} \, {\rm d}\mu.
\!$
where $\chi$ is support of $q(x)$. If you want to calculate $D_{\mathrm{KL}}(P\|Q)$, the problem is that the length of the vector $x$ in $P(x)$ and $Q(x)$ is random. I thought about the following:
$ D_{\mathrm{KL}}(P\|Q) = \sum_n \int_{\chi} P(x|n) \, \log \frac{P(x|n)}{Q(x|n)} \, {\rm d}\mu.
\!$
Is this right ? How about characterizing $P$ by joint pdf of number of jumps and the interrenewal distribution $p(x)$?
 A: I am not familiar with the general renewal process, so I will just use Poisson process between $t=0$ and $t=1$ as an example.
We may consider the random variables $(N, T)$, where $N$ is the number of jumps, and $T=(T_1,T_2,...,T_N)$ be the jump times. Similarly, let $(M,S)$ be the random variables from another Poisson process.
Now, using the formula
$D((N,T)\|(M,S))=D(N\|M) +D((T|N)\|(S|M))$
Now, the second part of the right hand side is the 'conditional relative entropy', which is defined as
$D((T|N)\|(S|M)) = \sum_{n=1}^\infty D((T|N=n)\|(S|M=n))
                 = \sum_{n=1}^\infty D(((T_1,T_2,...,T_n)|N=n)\|((S_1,S_2,...,S_n)|M=n)) $
this is well defined, because for each component, the $T$ and $S$ vectors are of the same dimension. And what is interesting about Poisson distribution is that, the conditional distribution (T|N=n), (S|M=n) are the same, regardless of their intensities, both of them are 'uniform' on $[0\ 1]$ ( remember we only consider Poisson process between $t=0$ and $t=1$ here). See the discussion in:
What does it mean for a Poisson point process $\Phi$'s points in $A$, conditioned on $\Phi(A)=k$ to be uniform?
So, the second part is $0$, we end up with
$D((N,T)\|(M,S))=D(N\|M)$
For general renewal process, I don't know whether similar conclusion holds, but the formula is still well defined.
