# Convergence of derivative of continuous Markov chain

Consider a continuous-time Markov process, described by:

$$\dot p = A_\theta\, p \tag{1}$$

where $$p \in \mathbb R^n$$ is a probability vector, and $$A_\theta \in \mathbb R^{n\times n}$$ is a transition matrix, of rank $$n-1$$, parameterized by $$\theta$$.

Given some initial distribution $$p_0$$, the probability vector in time is given by:

$$p(t,\theta) = e^{A_\theta t}\,p_0 \tag{2}$$

As $$t$$ increases, $$p(t,\theta)$$ converges to some stationary distribution $$p_\theta^\infty$$, the unique eigenvector of $$A_\theta$$ with null eigenvalue.

I want to study the Fisher information of $$p(t,\theta)$$ about $$\theta$$, so I look for its derivative:

\begin{align} \partial_\theta p(t,\theta) &= \partial_\theta (e^{A_\theta t}\,p_0) \\ &= t \int\limits_0^1 e^{\alpha A_\theta t} (\partial_\theta A_\theta) e^{-\alpha A_\theta t} d\alpha\ e^{A_\theta t}\, p_0 \\ &\equiv t\, B(t,\theta)\, p(t,\theta) \end{align} \tag{3}

I expected (¿maybe incorrectly?) the derivative $$\partial_\theta p(t,\theta)$$ to converge to some vector independent of $$t$$:

\begin{align} \lim_{t\rightarrow \infty} \partial_\theta p(t,\theta) &\stackrel{?}= \partial_\theta \lim_{t\rightarrow \infty} p(t,\theta) \tag{4}\\ \lim_{t\rightarrow \infty} t\, B(t,\theta)\, p(t,\theta) &\stackrel{?}= \partial_\theta p_\theta^\infty \tag{5} \end{align}

I know $$\partial_\theta p_\theta^\infty$$ is a non-zero finite vector.

## Question: Are Eqs. $$(4\text{ - }5)$$ correct?

If YES, how is the term $$t\, B(t,\theta)$$ not exploding as $$t\rightarrow \infty$$? I fail to see how $$\lim_{t\rightarrow \infty} B(t,\theta)\, p(t,\theta) \propto \frac{1}{t}$$.

If NO, where did I go wrong in my reasoning?

Thank you very much

This question made me think this problem from a different perspective, to try and work with the eigendecomposition of $$A_\theta$$.

Let $$\{p(\theta),v_2(\theta),\cdots,v_n(\theta)\}$$ be the eigenvectors of $$A_\theta$$, associated with the eigenvalues $$\{0,\lambda_2(\theta),\cdots,\lambda_n(\theta)\}$$, where $$\mathrm{Re}\{\lambda_j\}<0 \quad\forall_{j=2,\cdots,n}$$.

Write the initial distribution in the above eigenbasis, $$p_0 = p(\theta)+\sum_{j=2}^n c_j(\theta) v_j(\theta)$$, to obtain:

$$p(t,\theta)= e^{A_\theta t}p_0= \underbrace{\ p(\theta)\ }_{t-\mathrm{indep.}}+ \underbrace{\sum_{j=2}^n c_j(\theta) e^{\lambda_j(\theta)t} v_j(\theta)}_{t-\mathrm{dependent}}$$

As expected, $$p(t,\theta)$$ is converging to the stationary distribution $$p(\theta)$$. Turning to the derivative we again find that the $$t$$-dependent part vanishes as $$t\rightarrow \infty$$:

$$\partial p(t,\theta)= \underbrace{\ \partial_\theta p(\theta)\ }_{t-\mathrm{indep.}}+ \underbrace{\sum_{j=2}^n e^{\lambda_jt} \big(t\, \partial_\theta\lambda_j\, c_j\, v_j+ \partial_\theta c_j\, v_j + c_j\, \partial_\theta v_j\big)}_{t-\mathrm{dependent}}$$

Thus, $$\lim_{t\rightarrow \infty} \partial_\theta p(t,\theta) = \partial_\theta p(\theta) = \partial_\theta \lim_{t\rightarrow \infty} p(t,\theta)$$ Which means Eq.$$(4)$$ of the question is correct. I still don't know what to think of the term $$tB(t,\theta)$$ from Eq.$$(5)$$, but I guess I found a better way to frame the derivative, a way I am more comfortable with.