# Second derivative of Frobenius norm

Let $$A$$ be a real-valued matrix and $$f(A) := \lVert A \rVert_F = \sqrt{A:A} = \sqrt{\mbox{tr}(A^TA)}$$ be its Frobenius norm. What is $$\frac{\partial^2f}{\partial A^2}$$?

For the squared Frobenius norm, $$f^2$$, I found that $$\frac{\partial{f^2}}{\partial A}=2A$$ and $$\frac{\partial^2 f^2}{\partial A^2}=2 I\otimes I$$. For the case of interest I was able to compute $$\frac{\partial f}{\partial A}=A/\lVert A\rVert_F$$, however I am not able to deal with the square root in the denominator for the second derivative and would appreciate any advice.

• The gradient of scalar field $f^2$ is a function whose input and output are matrices. The 2nd derivative would be a $4$-dimensional matrix, for you need to differentiate every entry of the output matrix with respect to every entry of the input matrix. – Rodrigo de Azevedo May 25 '19 at 9:25

The matrix inner product $$\,A:B={\rm Tr}(A^TB)$$ can be used to write the Frobenius norm as \eqalign{ f^2 &= \|A\|_F^2 = A:A \cr } Find the differential and gradient of this expression. \eqalign{ 2f\,df &= 2A:dA \cr df &= \frac{A}{f}:dA \cr G=\frac{\partial f}{\partial A} &= \frac{A}{f} \cr } This matrix gradient can be vectorized. \eqalign{ g &= {\rm vec}(G) = \frac{{\rm vec}(A)}{f} = \frac{a}{f} \cr } Note the alternate expressions for the differential of the norm. \eqalign{ df &= G:dA \cr df &= g^Tda \cr } Now find the Hessian. \eqalign{ dg &= \frac{da}{f} - \frac{a}{f^2}\,df \cr &= \frac{da}{f} - \frac{a}{f^2}\,(g^Tda) \cr &= \frac{1}{f}\Big(I-gg^T\Big)\,da \cr H=\frac{\partial g}{\partial a} &= \frac{1}{f}\Big(I-gg^T\Big) \,\,= \frac{\partial^2f}{\partial a\,\partial a^T} \cr } The Hessian can also be written entirely in terms of $$a$$ \eqalign{ H &= \frac{1}{f^3}\Big(f^2I-aa^T\Big) } Without vectorization, the Hessian is a 4th order tensor, which can be written in component form as \eqalign{ H_{ijkl} &= \frac{\partial^2f}{\partial A_{ij}\,\partial A_{kl}} \cr &= \frac{1}{f^3}\Big(f^2\delta_{ik}\delta_{jl}-A_{ij}A_{kl}\Big) } Note the permuted order of the indices on the $$\delta$$ symbols.