For a function $f: A \rightarrow B$, Frechet differentiability tells us that we want to find a linear operator that satisfies

$$\lim_{H\rightarrow 0} \frac{||f[X+H] - f[X] - G[H]||}{||H||} = 0$$

This would mean that $G$ is a good approximation of the change in $f$ at $X$ for some small $H\in A$. That is for the operator $df: A \rightarrow B$

$$df(X)[H] = G[H]$$

Wikipedia says that $G$ is defined as the Frechet derivative of $f$ at $X$. But I have a litte trouble connecting this to the traditional notion of the derivative, where we have a fraction i.e. something like $\frac{dy}{dx}$.

For example, consider a standard formula from the matrix cookbook

$$\frac{dTr(XA)}{dX} = A^T$$

The Frechet differentiability definition gets me up to

$$dTr(XA)[H] = G[H] = Tr(HA).$$

What is then done is \begin{align} dTr(XA)[H] &= Tr(HA) \\ &= A^T :H, \end{align}

where we just use the notation $A:B = Tr(A^TB)$. What is the correct way to go from here to the conclusion that $$\frac{dTr(XA)}{dX} = A^T$$

and what does the LHS even represent exactly since it's not a fraction in the traditional sense?

More generally, what is involved in going from the differential form i.e. $df(X)[H] = G[H]$ to the derivative form $G = \frac{df(X)}{dX}$?


If $f: A\to B$ is Frechet differentiable then for all $X,V\in A$ the directional derivative

$$d_Vf(X)=\lim_{t\to 0}\frac{f(X+tV)-f(X)}{t}$$

exists and $d_Vf(X)=df(X)(V)$.

Now if $A=\mathbb R^{n\times n}$, $B=\mathbb R$ and $G = \frac{df(X)}{dX}$ then $G_{i,j}$ is just the directional derivative $d_{H_{i,j}}f(X)$ where $H_{i,j}$ is the matrix which has a $1$ on the $(i,j)$-th position and is zero otherwise. So the relation is


$\bullet$ Now consider the special map $f:\mathbb R^{n\times n}\to\mathbb R$ $X\mapsto Tr(XA)$. As this map is already linear we have $df(X)=f$ for all $X\in\mathbb R^{n\times n}$ so aplying the above relation yields


so $G=A^T$ which also can be computed directly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.