I've recently started to learn about Frechet derivatives and now have a simple example which I'm not sure if I've solved correctly. To be honest, I've only got a poor understanding of how it works so please assume I am starting at zero. Given $\delta = \sum_i \lambda_iX_i$ for some matrices $X_i$ and real numbers $\lambda_i$ and $N(X)$ being a linear transformation on $X$, find the Frechet derivative

$$\frac{\partial}{\partial \lambda_l}Tr(A\log(N(\delta)),$$

for some $l$.

Redoing my attempted solution after reading greg's answer. Would appreciate if anyone can comment on the correctness of it!

We have \begin{align} d Tr(A\log(N(\delta)) &= Tr(dA\log(N(\delta)) \\ &= Tr\left(\left(\int_0^{\infty}\frac{1}{1+tN(\delta)}A\mathrm{d}t\frac{1}{1+tN(\delta)}\right)dN(\delta)\right) \\ &= Tr\left(\left(\int_0^{\infty}\frac{1}{1+tN(\delta)}A\mathrm{d}t\frac{1}{1+tN(\delta)}\right)N(d\delta)\right) \\ &= Tr\left(\left(\int_0^{\infty}\frac{1}{1+tN(\delta)}A\mathrm{d}t\frac{1}{1+tN(\delta)}\right)N(X_k d\lambda_k)\right) \\ \end{align}

I have used the fact that $Tr$ and $N()$ are both linear operators and the solution posted here to write the function $d\ (A\log X)$. I believe that this cannot proceed further unless I can say something about $N()$ to rewrite $N(X_k d\lambda_k)$ as $ M(X_k) d\lambda_k$. If and only if I could do that step, then I have

\begin{align} d Tr(A\log(N(\delta)) &= Tr\left(\left(\int_0^{\infty}\frac{1}{1+tN(\delta)}A\mathrm{d}t\frac{1}{1+tN(\delta)}\right)M(X_k) d\lambda_k\right)\\ \frac{\partial}{\partial\lambda_k}Tr(A\log(N(\delta)) &= \left(\left(\int_0^{\infty}\frac{1}{1+tN(\delta)}A\mathrm{d}t\frac{1}{1+tN(\delta)}\right)M(X_k)\right)^T \end{align}


Define the matrix $$\eqalign{ Y &= \sum_{i=k}^N \lambda_kX_k = \lambda_kX_k \cr }$$ where the expression on the far RHS uses the index summation convention.

Now calculate the differential and derivative for the trace of a simple nonlinear function. $$\eqalign{ \phi &= {\rm Tr}\Big(Y^3\Big) \cr d\phi &= {\rm Tr}\Big(3Y^2\,dY\Big) = {\rm Tr}\Big(3Y^2X_k\,d\lambda_k\Big) \cr \frac{d\phi}{d\lambda_k} &= {\rm Tr}\Big(3Y^2X_k\Big) \cr }$$ A general function $f(Y)$ follows the same pattern. $$\eqalign{ \psi &= {\rm Tr}\Big(f(Y)\Big) \cr \frac{d\psi}{d\lambda_k} &= {\rm Tr}\Big(f'(Y)\,X_k\Big) \cr }$$ where $f'$ denotes the ordinary derivative of the function.

  • $\begingroup$ Thank you for this answer. Using this and your comments on my other question, I've redone the solution. I believe it can only proceed if I make an assumption about $N()$, as I have clarified in the edited question. If you wouldn't mind taking a moment to read it, can I check that this is the correct solution to the original problem I posed? $\endgroup$ – user1936752 Mar 17 at 22:23
  • 1
    $\begingroup$ It looks correct, I'm just not sure how practical it will prove to be. Evaluating an improper integral of a matrix-valued function is extremely tricky. If you just need a closed-form result, is an integral expression more useful than the original derivative expression? $\endgroup$ – greg Mar 19 at 0:32
  • $\begingroup$ Yeah, that's a good point. Thanks for the help :) $\endgroup$ – user1936752 Mar 19 at 10:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.