I had a couple of basic questions about the Frechet derivative.

1) The chain rule: If $F: X\rightarrow Y$ and $G: Y\rightarrow Z$, then $D(G\circ F)(x) = DG(F(x))DF(x)$. The left hand side has a composite map $D(G\circ F): X \rightarrow Z$ and is very clear to me but the right hand side is unclear. Should I be reading it as $DG(DF(x))$, where $DG$ is evaluated at $F(x)$ and $DF$ is evaluated at $x$?

Some others also present this with an additional $H$. That is, in the limit $H\rightarrow 0$, we have $D(G\circ F)[H]$ on the left hand side. The right hand side is written as $DG(F(X))DF(X)[DF(X)[H]]$ and I wasn't too sure how this worked.

2) If any function is linear, the Frechet derivative is the function itself. In this case, how does one express the right hand side of the chain rule when either $G$ is linear or $F$ is linear. Given my problems with the notation in the previous part, I'm not too sure how to rewrite the right hand side with $DG = G$ or $DF = F$.


Here's an example to which I know the solution so it becomes even clearer. It is known that $D Tr(A\log X) = Tr(A X^{-1} dX)$ for constant $A$ but how do I use the chain rule to get this? My attempt is below

$$D Tr(A\log X) = D Tr(A\log X)D(A\log X) = DTr(A\log X) AX^{-1}dX = ??$$

Clearly, I don't know exactly how to use the rule correctly. Please let me know how to proceed. Thank you

  • 1
    $\begingroup$ Your example starts from a false premise, since $$d\,{\rm Tr}(A\log(X)) \ne {\rm Tr}(AX^{-1}dX)$$ $\endgroup$ – greg Mar 15 at 17:24
  • $\begingroup$ @greg, thanks for your comment. This was posted as a comment (which stated that $\frac{d Tr(A\log(X))}{dX} = X^{-T}A^T)$ to my linked question and seemed correct to me. What is the mistake? $\endgroup$ – user1936752 Mar 15 at 17:30
  • 3
    $\begingroup$ Instead of a complicated function like $\log(X)$, try taking the derivative of a simple function like $X^3$. In the space of a few short lines you should be able to convince yourself that $$\eqalign{ d\,{\rm Tr}(X^3) &= {\rm Tr}(3X^2dX) \cr d\,{\rm Tr}(AX^3) &\ne {\rm Tr}(3AX^2dX) \cr }$$ Once you understand the reason for that last inequality, you'll understand why the proposed log-formula is flawed. [Hint: The $A$-matrix messes up everything.] $\endgroup$ – greg Mar 15 at 18:57
  • $\begingroup$ @greg that was a really insightful comment, thank you. I see that $d Tr(AX^3) = Tr((X^2A + XAX + AX^2) dX)$. It's helped put your other answer to my question in much better context too. Really appreciate the help! $\endgroup$ – user1936752 Mar 16 at 21:33

1) It's not clear to you since what you wrote is incorrect. It would be correct however you wrote $D(G \circ F)(x): X \rightarrow Z$ (i.e, what you had but evaluated at some x in domain of $F$), or you could say $D(G \circ F):X \rightarrow \mathcal{L}(X;Z)$ , and the RHS is a composition, so should have a '$\circ$' in between, i.e $DG(F(x)) \circ DF(x)$. With this you should now be able to see what is going on with $H$, it's an element of $X$ and so we are mapping $H$ to $D(G \circ F)(x)(H) \in Z$

2) If $F: X \rightarrow Y$ is linear, then $DF(x): X \rightarrow Y$ is equal to F (that is, F=DF(x)), notice that derivative is being evaluated at a point in $X$ whilst the original function is not, this is crucial in understanding the Frechet derivative.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.