Let
- Let $\|M\| := \sigma_{max}(M)$, i.e. the spectral norm of $M$.
- $\langle X,Y \rangle = \text{Trace}(X^TY)$. Inner product of two matrices.
How to prove the title:
If $X = U\Sigma V^T$ and $\|Y\|=1$, why $Y = UV^T$ maximizes $\langle X,Y\rangle$
As a consequence:
- We know since $Y=UIV^T$, so $\sigma_{max}(Y) = 1$.
- Trace$(X^TY) = $trace$(\Sigma)$ = $\|X\|_*$, which is the nuclear norm.
The above is the key to prove that spectral norm and nuclear norm are dual.
(Also if you are interested in this proof, "Show that the dual norm of the spectral norm is the nuclear norm" is a good reference.)
Thank @angryavian. The proof is the following:
I met another problem: how to show the last two equalities?