5
$\begingroup$

I have $x=\exp(At)$ where $A$ is a matrix. I would like to find derivative of $x$ with respect to each element of $A$. Could anyone help with this problem?

$\endgroup$
  • $\begingroup$ ...you mean derivative of every matrix element of $x$ with respect to $t$? $\endgroup$ – draks ... May 20 '15 at 12:55
11
$\begingroup$

Considering the expression $x = \exp(tA)$ I can think of two derivatives.

First, the derivative with respect to the real variable $t$ of the matrix-valued function $t \mapsto \exp(tA)$. Here the result is easily derived from direct calculation of the series definition of the matrix exponential: \begin{align} \frac{d}{dt} \exp(tA) &= \frac{d}{dt} \left[ I+tA+\frac{1}{2}t^2A^2+\frac{1}{3!}t^3A^3+ \cdots\right] \\ &= A+tA^2+\frac{1}{2}t^2A^3+ \cdots \\ &= A\exp(tA) \end{align} Thus, $\frac{d}{dt} \exp(tA) = A\exp(tA)$. (Edited to fix typo)

Second, we can differentiate with respect to the component $A_{ij}$ of $A = \sum A_{ij} E_{ij}$ where $E_{ij}=e_ie_j^T$ is the matrix which is zero except in the $ij$-th spot where there is a $1$. In other words, $(E_{ij})_{kl} = \delta_{ik}\delta_{jl}$. I'll look at the derivative as a directional derivative essentially: calculate the difference along the $E_{ij}$ direction: considering $f(t,A)=\exp(tA)$ $$ \frac{\partial f}{\partial A_{ij}}= \lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(tA)-\exp(t(A+hE_{ij}))\right]$$ I expect this can be simplified.

Ok, the matrix exponential satisfies the Baker-Campbell-Hausdorf relation: $$ \exp(A)\exp(B) = \exp\left(A+B+ \frac{1}{2}[A,B] + \cdots\right)$$ From this we derive the Zassenhaus formula, $$ \exp(A+B) = \exp(A)\exp(B)\exp\left(-\frac{1}{2}[A,B] + \cdots\right) $$ I'll use this to simplify $\exp( t(A+hE_{ij})) = \exp\left(tA+ thE_{ij}\right)$ $$ \exp\left(tA+ thE_{ij}\right) = \exp(tA)\exp\left( thE_{ij}\right) \exp\left( -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right)$$ hence $$ \exp\left(tA+ thE_{ij}\right) = \exp(tA)\exp\left( thE_{ij} -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right)$$ where I am omitting terms with $h^2,h^3,\dots$ as those vanish in the limit and I am also omitting terms with nested commutators of $A$ so the answer below is just the first couple terms in an infinite series flowing from the BCH relation. \begin{align} \frac{\partial f}{\partial A_{ij}}&= \lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(tA)-\exp(t(A+hE_{ij}))\right] \\ &=\lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(tA)-\exp(tA)\exp\left( thE_{ij} -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right) \right] \\ &=\exp(tA)\lim_{h \rightarrow 0}\frac{1}{h} \left[I-\exp\left( thE_{ij} -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right) \right] \\ &=\exp(tA)\lim_{h \rightarrow 0}\frac{1}{h} \left[I-I-thE_{ij} +\frac{1}{2}[tA,thE_{ij}]+ \cdots \right] \\ &=\exp(tA)\left[-tE_{ij} +\frac{t^2}{2}[A,E_{ij}]+ \cdots \right]. \end{align} Note the terms linear in $h$ do survive the limit and there are such terms (indicated by the $+ \cdots$) stemming from $[tA,[tA,hE_{ij}]]$ and $[tA,[tA,[tA,hE_{ij}]]]$ etc. Now, you can calculate: $[A,E_{ij}] = \sum_{k=1}^n \left(A_{ki}E_{kj}-A_{jk}E_{ik} \right)$ so, $$ \frac{\partial f}{\partial A_{ij}} = \exp(tA) \left[-tE_{ij}+ \frac{t^2}{2}\left(A_{ki}E_{kj}-A_{jk}E_{ik} +\cdots \right)\right]$$ For what it's worth, you can simplify the nested commutator: $$ [A,[A,E_{ij}]] = \sum_{k,l=1}^n \left( A_{lk}A_{ki}E_{lj}-2A_{ki}A_{jl}E_{kl}+A_{jl}A_{lk}E_{ik} \right)$$ Or, in Einstein notation, $$ [A,[A,E_{ij}]] = (A^2)_{li}E_{lj}-2A_{ki}A_{jl}E_{kl}+(A^2)_{jk}E_{ik}.$$ Anyway, I hope this helps. Notice if $i=j$ and $A$ is diagonal or if simply $[A,E_{ij}]=0$ then we obtain: $$ \frac{\partial }{\partial A_{ij}} \exp(tA) = -t\exp(tA)E_{ij}.$$ which makes me think I've made a sign-error somewhere*. If someone sees it and messages me about it I would be happy.

*The sign-error lies in the definition equation of $\frac{\partial f}{\partial A_{ij}}$. It should be $$ \frac{\partial f}{\partial A_{ij}}= \lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(t(A+hE_{ij})) - \exp(tA)\right].$$

$\endgroup$
  • $\begingroup$ Thank you for the answer James. I am looking for differentiate with respect to the component Aij. I would be waiting for a more simplification. $\endgroup$ – darwin rajpal May 20 '15 at 14:06
  • $\begingroup$ @HAyAs thanks! Excellent edit. I wish more edits were substantial like yours. $\endgroup$ – James S. Cook Oct 17 '16 at 2:14
4
$\begingroup$

If $A\in{\mathbb R}^{n\times n}$, then you can use Higham's "Complex Step Approximation" to calculate each component $$ \frac {\partial f} {\partial A_{jk}} = {\rm Im}\bigg(\frac{f(A+ihE_{jk})}{h}\bigg) $$ where $f(A)={\rm exp}(tA)$ and $h=10^{-20}$.

$\endgroup$
  • $\begingroup$ Somebody downvoted this post; one wonders why... Indeed, the presented numerical method is very effective. $\endgroup$ – loup blanc Oct 17 '16 at 10:00
0
$\begingroup$

Edit: see comment by loup blanc below for explanation of why I am incorrect.

The top answer that $D_{A_{ij}}\exp(A)=\exp(A) E_{ij}$ seems incorrect. In particular, I think the assumption

\begin{equation} \exp(A)\exp(B) = \exp\left(A+B+ \frac{1}{2}[A,B] + \cdots\right) \\ \Rightarrow \exp(A+B) = \exp(A)\exp(B)\exp\left(-\frac{1}{2}[A,B] + \cdots\right) \end{equation}

is wrong. Maybe I just can't see it, but I think the logic assumes $\exp(A+B)=\exp(A)\exp(B)$ which is not true for general matrices.

Otherwise, answers from elsewhere seem to provide correct answers: Derivative of the matrix exponential with respect to its matrix argument

$\endgroup$
  • $\begingroup$ This appears to be commentary on other answers and a link to other similar answers, rather than a direct answer to the question being asked. If you think that the other answers are wrong, you should either reply to those answers and hope that the original authors reappear to clarify, or post a new answer that is complete (and doesn't simply link to other questions). $\endgroup$ – Xander Henderson Nov 8 '17 at 4:14
  • $\begingroup$ Your "top answer" is correct because it is assumed that $A,E_{i,j}$ commute. Yet, the second formula is badly written. By Zassenhaus, $e^{A+B}=e^{A}e^{B}e^{-1/2[A,B]}e^{1/3[B,[A,B]]+1/6[A,[A,B]]}e^{\cdots}\cdots$ $\endgroup$ – loup blanc Nov 11 '17 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.