# Can the transition probabilities of an inhomogeneous Markov chain be written as an exponential?

If $Z_t$ is a homogeneous continuous-time Markov chain with finite state space $E=\{1,\ldots,p\}$, transition matrices $(P(t))$ and intensity matrix $Q$, it holds that $$P(t) = \exp(tQ),$$ see for example Chapter 2 of the book "Markov chains" by J. R. Norris. I am interested in whether this can be extended to the inhomogeneous case, and if no, why not. The absence of references indicating any sort of expression for the transition matrices for inhomogeneous chains using matrix exponentials would suggest that a simple extension is not possible.

Before formulating my precise question, let me express a few facts from the theory of inhomogeneous chains. In the in homogeneous case, the intensities are time-dependent. Consider given a continuous function $t\mapsto Q(t)$ of $p\times p$ $Q$-matrices (meaning that the rows sum to one). Assume that we are given a set of transition matrices $P(s,t)$ for $0\le s\le t$ such that $$\lim_{h\to 0}\frac{1}{h}(P(t,t+h)-I_p) = Q(t)$$ where $I_p$ is the $p\times p$ identity matrix. It then holds that $$\frac{d}{ds}P(s,t) = -Q(s)P(s,t)$$ and $$\frac{d}{dt}P(s,t) = P(s,t)Q(t)$$ hold. These are the Kolmogorov backward and forward equations, respectively. This follows from Feller's 1940 paper "On the integro-differential equations of purely discontinuous Markoff processes".

Now consider the mapping $$\tilde{P}(s,t) = \exp\left(\int_s^t Q(u) du\right).$$ It would be natural to conjecture that $P(s,t) = \tilde{P}(s,t)$. By the matrix chain rule, it at least holds that $$\frac{d}{ds}\tilde{P}(s,t) = -Q(s)\tilde{P}(s,t),$$ so $\tilde{P}$ satisfies the backward equation. However, by the same reasoning, we appear also to have $$\frac{d}{dt}\tilde{P}(s,t) = Q(t)\tilde{P}(s,t),$$ which is not the same as the backward equation. The above motivates several questions:

-First, assuming that uniqueness holds for the backward equation (which appears to also have been settled by Feller's 1940 paper, in his Theorem 3), the above would seem to imply $P(s,t) = \tilde{P}(s,t)$. Is this really correct?

-Is it possible to transform the forward equation into the equation solved by the exponential in the above? If yes, how?

-If the equality between $P(s,t)$ and $\tilde{P}(s,t)$ does not hold, where is the error in the above reasoning, and is there some intuitive reason why the result does not hold?

• How do you differentiate $\bar P(s,t)$ exactly? – Did Jul 22 '14 at 14:11
• My thought was that for fixed $t\ge0$, $\tilde{P}(s,t) = \exp(f(s))$ with $f(s) = \int_s^t Q(u) du$, and then the chain rule should yield $d/ds \tilde{P}(s,t) = f'(s)\exp(\tilde{P}(s,t))$. On second thought, though, now that you mention it, I see that while $\exp'(tA) = A\exp(tA)$, trying to apply the chain rule as I do seems to be unfounded. – Alexander Sokol Jul 22 '14 at 14:21
• Indeed, it happens that $$\left.\frac{\mathrm d}{\mathrm dt}\mathrm e^{A+tB}\right|_{t=0}=\sum_{i\geqslant0}\sum_{k\geqslant0}\frac{A^iBA^k}{(i+k+1)!},$$ which is neither $B\mathrm e^A$ not $\mathrm e^AB$ in general (that is, except if $A$ and $B$ commute). – Did Jul 22 '14 at 14:24
• Well, it seems to me that this explains why the exponential formula does not extend to the inhomogeneous case. Thank you! – Alexander Sokol Jul 22 '14 at 14:45

As explained in the comments, differentiating exponentials of matrices such as $\bar P(s,t)$ is slightly more complicated than in the scalar case since, quite generally, $$\left.\frac{\mathrm d}{\mathrm dt}\mathrm e^{A+tB}\right|_{t=0}=\sum_{i\geqslant0}\sum_{k\geqslant0}\frac{A^iBA^k}{(i+k+1)!},$$ which is neither $B\mathrm e^A$ not $\mathrm e^AB$ in general (that is, except if $A$ and $B$ commute). Thus, if $t\mapsto A(t)$ is differentiable, then $$\frac{\mathrm d}{\mathrm dt}\mathrm e^{A(t)}=\sum_{i\geqslant0}\sum_{k\geqslant0}\frac{A(t)^iA'(t)A(t)^k}{(i+k+1)!}.$$ Finally, if $A(t)$ and $A(s)$ do commute for every $t\ne s$, then $A(t)$ and $A'(t)$ commute for every $t$ and, in this specific case, one gets indeed $$\frac{\mathrm d}{\mathrm dt}\mathrm e^{A(t)}=A'(t)\mathrm e^{A(t)}=\mathrm e^{A(t)}A'(t).$$