Frobenius and operator-2 norm

I have been studying about norms and for a given matrix A, I haven't been able to understand the difference between Frobenius norm $$||A||_F$$ and operator-2 norm $$|||A|||_2$$. Can someone help me understand the difference between them?

• Maybe this question is useful. It is not exactly your question, but some answers mention the difference between these norms. – Ernie060 Nov 13 '18 at 15:40

$$\|A\|_F$$ is $$\|\operatorname{vec}(A)\|_2$$, the Euclidean norm of the vector $$\operatorname{vec}(A)$$ obtained by stacking the columns of $$A$$ one above the other. So, you just reshape $$A$$ into a vector and take its Euclidean norm. The Frobenius norm of $$A$$ can also be expressed as $$\sqrt{\operatorname{tr}(A^\ast A)}$$, because each diagonal entry of $$A^\ast A$$ is the squared Euclidean norm of a row of $$A$$.

$$\|A\|_2$$, in contrast, is the maximum possible Euclidean norm of $$Av$$ for a unit vector $$v$$. Since $$A$$ a linear operator that operates on $$v$$, we call it an operator norm. And as we consider the $$2$$-norm of $$Av$$, the matrix norm $$\|A\|_2$$ is also called an induced norm. Had the vector $$p$$-norm been used in place of the $$2$$ norm, the resulting matrix norm $$\|A\|_p=\max_{\|v\|_p=1}\|Av\|_p$$ is called an operator/induced $$p$$-norm.

For example, $$\|I_2\|_F=\left\|\pmatrix{1\\ 0\\ 0\\ 1}\right\|_2=\sqrt{2}\ \text{ but } \ \|I_2\|_2=\max_{\|v\|_2=1}\|I_2v\|_2=\max_{\|v\|_2=1}\|v\|_2=1.$$

Both the operator $$2$$-norm (but not other induced $$p$$-norms) and Frobenius norm are unitarily invariant, i.e. $$\|UAV\|_F=\|A\|_F$$ and $$\|UAV\|_2=\|A\|_2$$ whenever $$U,V$$ are unitary matrices. Therefore, by singular value decomposition, we always have $$\|A\|_F = \sqrt{\sum_i\sigma_i(A)^2}\ge\sigma_1(A)=\|A\|_2$$ and the two matrix norms are equal only when $$\sigma_2(A)=\cdots=\sigma_n(A)=0$$, i.e. when $$\operatorname{rank}(A)\le1$$.

The operator $$2$$-norm is also confusingly called spectral norm in the literature, but as we have seen in the above, $$\|A\|_2$$ is the largest singular value of $$A$$. It is not really about the spectrum of $$A$$. The term probably originated from considering the spectrum of $$A^\ast A$$.

• For the induced norm are we taking the maximum over all possible unit vectors $v$ (or is $v$ specified before hand?). If it is the maximum over all possible v, could you perhaps give an example of calculating this when $A$ is not an identity matrix? – user106860 Nov 13 '18 at 16:41
• @user106860 Of course $v$ is not fixed. Since $\|A\|_2=\sigma_1(A)$, all examples can be reduced to those where $A$ are diagonal matrices. So, e.g. when $A=\operatorname{diag}(a,b)$ where $a\ge b$, we have $\|A\|_F=\sqrt{a^2+b^2}$ and $\|Av\|_2$ is maximised for a unit vector $v$ when $v=(1,0)^T$. – user1551 Nov 13 '18 at 16:51
• Thank you. Also a question on notation: about calling the norm the induced norm, what should I think of as "inducing" the norm? (It shouldn't be $v$ because $v$ is not fixed. So that just leaves $A$ or $Av$?) – user106860 Nov 13 '18 at 17:09
• @user106860 Every vector $p$-norm gives rise to a matrix $p$-norm, so the term means the matrix norm is induced by the vector norm. – user1551 Nov 13 '18 at 18:04