Derivative of matrix exponential w.r.t. to each element of the matrix I have $x= \exp(At)$ where $A$ is a matrix. I would like to find derivative of $x$ with respect to each element of $A$. Could anyone help with this problem?
 A: If $A\in{\mathbb R}^{n\times n}$, then you can use Higham's "Complex Step Approximation" to calculate each component 
$$
  \frac {\partial f} {\partial A_{jk}} = {\rm Im}\bigg(\frac{f(A+ihE_{jk})}{h}\bigg)
$$
where $f(A)={\rm exp}(tA)$  and $h=10^{-20}$. 
A: According to Derivatives of the Matrix Exponential and Their Computation (who reference Karplus, Schwinger, Feynmann, Bellman and Snider) the derivative can be expressed as the linear map (i.e. Fréchet derivative)
$$\frac{\rm{d} e^{At}}{\rm{d} A} = \Big(V\longmapsto\int_{0}^t e^{A(t-\tau)}Ve^{A\tau}\,\rm{d}\tau\Big)$$
A: Considering the expression $x = \exp(tA)$ I can think of two derivatives.
First, the derivative with respect to the real variable $t$ of the matrix-valued function $t \mapsto \exp(tA)$. Here the result is easily derived from direct calculation of the series definition of the matrix exponential:
\begin{align} \frac{d}{dt} \exp(tA) &= \frac{d}{dt} \left[ I+tA+\frac{1}{2}t^2A^2+\frac{1}{3!}t^3A^3+ \cdots\right] \\
&= A+tA^2+\frac{1}{2}t^2A^3+ \cdots \\
&= A\exp(tA) 
\end{align}
Thus, $\frac{d}{dt} \exp(tA) = A\exp(tA)$.   (Edited to fix typo)
Second, we can differentiate with respect to the component $A_{ij}$ of $A = \sum A_{ij} E_{ij}$ where $E_{ij}=e_ie_j^T$ is the matrix which is zero except in the $ij$-th spot where there is a $1$. In other words, $(E_{ij})_{kl} = \delta_{ik}\delta_{jl}$. I'll look at the derivative as a directional derivative essentially: calculate the difference along the $E_{ij}$ direction: considering $f(t,A)=\exp(tA)$
$$ \frac{\partial f}{\partial A_{ij}}= \lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(t(A+hE_{ij}))-\exp(tA)\right]$$
I expect this can be simplified.
Ok, the matrix exponential satisfies the Baker-Campbell-Hausdorf relation:
$$ \exp(A)\exp(B) = \exp\left(A+B+ \frac{1}{2}[A,B] + \cdots\right)$$
From this we derive the Zassenhaus formula,
$$ \exp(A+B) =  \exp(A)\exp(B)\exp\left(-\frac{1}{2}[A,B] + \cdots\right) $$
I'll use this to simplify $\exp( t(A+hE_{ij})) = \exp\left(tA+ thE_{ij}\right)$
$$ \exp\left(tA+ thE_{ij}\right) = \exp(tA)\exp\left( thE_{ij}\right)
\exp\left( -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right)$$
hence
$$ \exp\left(tA+ thE_{ij}\right) = \exp(tA)\exp\left( thE_{ij} -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right)$$
where I am omitting terms with $h^2,h^3,\dots$ as those vanish in the limit and I am also omitting terms with nested commutators of $A$ so the answer below is just the first couple terms in an infinite series flowing from the BCH relation.
\begin{align} \frac{\partial f}{\partial A_{ij}}&= -\lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(tA)-\exp(t(A+hE_{ij}))\right] \\
&=-\lim_{h \rightarrow 0}\frac{1}{h} \left[\exp(tA)-\exp(tA)\exp\left( thE_{ij} -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right) \right] \\
&=-\exp(tA)\lim_{h \rightarrow 0}\frac{1}{h} \left[I-\exp\left( thE_{ij} -\frac{1}{2}[tA,thE_{ij}]+ \cdots\right) \right] \\
&=-\exp(tA)\lim_{h \rightarrow 0}\frac{1}{h} \left[I-I-thE_{ij} +\frac{1}{2}[tA,thE_{ij}]+ \cdots \right] \\
&=-\exp(tA)\left[-tE_{ij} +\frac{t^2}{2}[A,E_{ij}]+ \cdots \right]. 
\end{align}
Note the terms linear in $h$ do survive the limit and there are such terms (indicated by the $+ \cdots$) stemming from $[tA,[tA,hE_{ij}]]$ and $[tA,[tA,[tA,hE_{ij}]]]$ etc.
Now, you can calculate: $[A,E_{ij}] = \sum_{k=1}^n \left(A_{ki}E_{kj}-A_{jk}E_{ik} \right)$ so,
$$ \frac{\partial f}{\partial A_{ij}} = -\exp(tA) \left[-tE_{ij}+ \frac{t^2}{2}\left(A_{ki}E_{kj}-A_{jk}E_{ik} +\cdots \right)\right]$$
For what it's worth, you can simplify the nested commutator:
$$ [A,[A,E_{ij}]] = \sum_{k,l=1}^n \left( A_{lk}A_{ki}E_{lj}-2A_{ki}A_{jl}E_{kl}+A_{jl}A_{lk}E_{ik} \right)$$
Or, in Einstein notation,
$$ [A,[A,E_{ij}]] = (A^2)_{li}E_{lj}-2A_{ki}A_{jl}E_{kl}+(A^2)_{jk}E_{ik}.$$
Anyway, I hope this helps. Notice if $i=j$ and $A$ is diagonal or if simply $[A,E_{ij}]=0$ then we obtain:
$$ \frac{\partial }{\partial A_{ij}} \exp(tA) = t\exp(tA)E_{ij}.$$
(I fixed the sign-errors, sorry for all weridly placed minus signs: 9-18-21).
A: Edit: see comment by loup blanc below for explanation of why I am incorrect.
The top answer that $D_{A_{ij}}\exp(A)=\exp(A) E_{ij}$ seems incorrect. In particular, I think the assumption
\begin{equation}
\exp(A)\exp(B) = \exp\left(A+B+ \frac{1}{2}[A,B] + \cdots\right) \\
\Rightarrow
\exp(A+B) =  \exp(A)\exp(B)\exp\left(-\frac{1}{2}[A,B] + \cdots\right) 
\end{equation}
is wrong. Maybe I just can't see it, but I think the logic assumes $\exp(A+B)=\exp(A)\exp(B)$ which is not true for general matrices.
Otherwise, answers from elsewhere seem to provide correct answers: Derivative of the matrix exponential with respect to its matrix argument
A: I arrive at the following. The ij element of $e^A$ is $$e^A_{ij} = \sum_{n=0}^\infty \frac{1}{n!} A_i^{k_1} ... A_{k_{n-1}j} ~.$$ Each matrix element can be seen as an independent variable, so the derivative toward $A^{kl}$ is $$e^A_{ij} = \sum_{n=0}^\infty \frac{1}{n!}  \sum_{p=0}^{n-1} A^p_{ik} A^{n-1-p}_{lj} ~.$$
For my purpose $t$ is less relevant. It can easily be added in without changing my result.
