When differentiating with respect to a matrix, is it possible to rewrite the derivative operator via some transformation so that you're differentiating with respect to the diagonal eigenvalue matrix instead? I.e., is something like this possible? \begin{equation}\frac{\partial }{\partial A}=Q'\frac{\partial}{\partial \Lambda}Q\end{equation} Where $Q$ is some transformation and $\Lambda$ is the diagonal matrix of eigenvalues. The reason I ask is because I'm working on something like this \begin{equation}f(\Lambda)=e^{\sum_{i}\lambda_{i}^{2}v_{i}}=e^{\mathrm{tr} \Lambda^{2}V}\end{equation} I can rewrite this as \begin{equation}f(A)=e^{\mathrm{tr}UA^{2}U^{\dagger}V}=e^{\mathrm{tr}A^{2}U^{\dagger}VU}\end{equation} I'm not sure if differentiating with respect to $A$ is ok because of the unitary transformations $U$ which technically depend on $A$.


Let $f:M_n(\mathbb{R})\rightarrow \mathbb{R}$ be a $C^1$ function. Let $A$ be a diagonalizable matrix $A=PDP^{-1}$ with $A=[a_{i,j}],P=[p_{i,j}],P^{-1}=[p'_{i,j}]$ and $P$ is a fixed invertible matrix. Let $(E_{i,j})$ be the canonical basis of $M_n(\mathbb{R})$. Then $\dfrac{∂D}{∂a_{i,j}}=P^{-1}E_{i,j}P\;$ ($D$ is diagonal but a perturbation of $D$ is not). We consider the scalar product on $M_n(\mathbb{R})$ defined by $<U,V>=trace(UV^T)=\sum_{k,l}u_{k,l}v_{k,l}$ where $U=[u_{k,l}],V=[v_{k,l}]$.

Proposition: $\dfrac{∂f}{∂a_{i,j}}=< \dfrac{∂f}{∂D},\dfrac{∂D}{∂a_{i,j}}>$.

Proof: $\dfrac{∂f}{∂a_{i,j}}=\sum_{k,l}\dfrac{∂f}{∂d_{k,l}}\dfrac{∂d_{k,l}}{∂a_{i,j}}=\sum_{k,l}(\dfrac{∂f}{∂D})_{k,l}(\dfrac{∂D}{∂a_{i,j}})_{k,l}$

For instance(the one of user?? few days ago), let $A=\begin{pmatrix}1&1\\0&-1\end{pmatrix},D=\begin{pmatrix}1&0\\0&-1\end{pmatrix},P=\begin{pmatrix}1&-1/2\\0&1\end{pmatrix}$ and let $f(A)=\sup(spectrum(A))$. Then $\dfrac{∂f}{∂A}=\begin{pmatrix}1&0\\1/2&0\end{pmatrix},\dfrac{∂f}{∂D}=\begin{pmatrix}1&0\\0&0\end{pmatrix}$. The formula above gives the link between these $2$ derivatives. Recall that $f$ is $C^1$ because the $\sup $ is reached by a sole eigenvalue.

  • $\begingroup$ Could you elaborate on the last part? If $A$ is fixed, how do you define an operation like $\frac{\partial}{\partial A}$? Is it just by adding a perturbation in analogy to a standard derivative? $\endgroup$ – TeeJay Nov 20 '13 at 20:58
  • $\begingroup$ My notation is not very good. $\dfrac{∂f}{∂A}=(\dfrac{∂f}{∂a_{i,j}})_{i,j}$ and, for instance, $\dfrac{∂f}{∂a_{1,1}}=\lim_{x\rightarrow 0}\dfrac{f(\begin{pmatrix}1+x&1\\0&-1\end{pmatrix})-f(A)}{x}$. $\endgroup$ – loup blanc Nov 20 '13 at 22:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.