# Second Partial Derivative Test for a Matrix Valued Function

Let $$X\in\mathbb{R}^{n\times m}$$ and $$f(X)=\|X^TX-I_m\|^2_F$$ (the Frobenius matrix norm). I was able to derive the derivative w.r.t X that is $$df_X(A)=\langle 4 (X^TX-I_m)X^T,A\rangle$$ (the Frobenius inner product). Therefore, $$\nabla_XF=4 (X^TX-I_m)X^T$$, and $$F$$ has two critical points at $$X=0$$ and $$X^TX=I_m$$.

I want to show that $$X=0$$ is a local maximum (not sure maybe it is a saddle point). The Hessian becomes a 4th order tensor and very cumbersome to derive. I tried to find the first order approximation of the gradient which turned out to be $$\nabla_XF(X+H)\simeq\nabla_XF(X)+4(H^TXX^T+X^THX^T+X^TXH^T-H^T).$$ However, I don't know how to use this to examine the function at $$X=0$$. How can I see what type of critical point $$X=0$$ is?

• What is Fibbonacci norm? Do you mean Frobenius norm? Nov 10 '18 at 10:44
• There are other critical points than $X=0$ and $X^TX=I$. Since $(X^TX-I)X^T=0$ if and only if $(X^TX-I)X^TX=0$, the critical points are those matrices whose only singular values are $0$ or $1$. In the fashion of method 2 in loup blanc's answer, one can show that a critical point $X$ is a local min if all its singular values are equal to $1$ (i.e. if $X$ has orthonormal columns or rows), and it is a saddle point if both $0$ and $1$ are its singular values. Nov 10 '18 at 16:50

Method 1 (with the Hessian): the quadratic form is

$$D^2f_X(H,H)=4tr(H^TH(X^TX-I_m)+H^TXH^TX+H^TXX^TH)$$

where $$H\in M_{n,m}$$.

When $$X=0$$, $$D^2f_0(H,H)=-4tr(H^TH)$$, that is $$<0$$ when $$H\not=0$$. Then our quadratic form is $$<0$$ and, since $$X=0$$ is a critical point of $$f$$, it is also a local maximum of $$f$$.

Method 2. Let $$spectrum(X^TX)=(\sigma_i^2)$$ (the singular values of $$X$$). Then

$$f(X)=tr((X^TX-I)^2)=\sum_{i\leq m}(\sigma_i^2-1)^2$$. When $$X$$ is in a neighborhood of $$0$$, then the $$\sigma_i^2$$ are small and

$$f(X)\approx \sum_{i\leq m}(1-2\sigma_i^2)=m-2\sum_i\sigma_i^2\leq m=f(0)$$ and we are done.

Remark. Of course, there are other critical points than $$X=0$$ and $$X$$ pseudo orthogonal.

• Thanks for the answer! Would you please explain further the final remark. What are the critical points other than $X=0$ and $X^TX=I_m$? are those the ones that $X^TXX^T=X^T$? If so how can we use method 1 to investigate those too? Nov 10 '18 at 15:16