I have been studying about norms and for a given matrix A, I haven't been able to understand the difference between Frobenius norm $||A||_F$ and operator-2 norm $|||A|||_2$. Can someone help me understand the difference between them?

  • $\begingroup$ Maybe this question is useful. It is not exactly your question, but some answers mention the difference between these norms. $\endgroup$ – Ernie060 Nov 13 '18 at 15:40

$\|A\|_F$ is $\|\operatorname{vec}(A)\|_2$, the Euclidean norm of the vector $\operatorname{vec}(A)$ obtained by stacking the columns of $A$ one above the other. So, you just reshape $A$ into a vector and take its Euclidean norm. The Frobenius norm of $A$ can also be expressed as $\sqrt{\operatorname{tr}(A^\ast A)}$, because each diagonal entry of $A^\ast A$ is the squared Euclidean norm of a row of $A$.

$\|A\|_2$, in contrast, is the maximum possible Euclidean norm of $Av$ for a unit vector $v$. Since $A$ a linear operator that operates on $v$, we call it an operator norm. And as we consider the $2$-norm of $Av$, the matrix norm $\|A\|_2$ is also called an induced norm. Had the vector $p$-norm been used in place of the $2$ norm, the resulting matrix norm $\|A\|_p=\max_{\|v\|_p=1}\|Av\|_p$ is called an operator/induced $p$-norm.

For example, $$ \|I_2\|_F=\left\|\pmatrix{1\\ 0\\ 0\\ 1}\right\|_2=\sqrt{2}\ \text{ but } \ \|I_2\|_2=\max_{\|v\|_2=1}\|I_2v\|_2=\max_{\|v\|_2=1}\|v\|_2=1. $$

Both the operator $2$-norm (but not other induced $p$-norms) and Frobenius norm are unitarily invariant, i.e. $\|UAV\|_F=\|A\|_F$ and $\|UAV\|_2=\|A\|_2$ whenever $U,V$ are unitary matrices. Therefore, by singular value decomposition, we always have $$ \|A\|_F = \sqrt{\sum_i\sigma_i(A)^2}\ge\sigma_1(A)=\|A\|_2 $$ and the two matrix norms are equal only when $\sigma_2(A)=\cdots=\sigma_n(A)=0$, i.e. when $\operatorname{rank}(A)\le1$.

The operator $2$-norm is also confusingly called spectral norm in the literature, but as we have seen in the above, $\|A\|_2$ is the largest singular value of $A$. It is not really about the spectrum of $A$. The term probably originated from considering the spectrum of $A^\ast A$.

| cite | improve this answer | |
  • $\begingroup$ For the induced norm are we taking the maximum over all possible unit vectors $v$ (or is $v$ specified before hand?). If it is the maximum over all possible v, could you perhaps give an example of calculating this when $A$ is not an identity matrix? $\endgroup$ – user106860 Nov 13 '18 at 16:41
  • 1
    $\begingroup$ @user106860 Of course $v$ is not fixed. Since $\|A\|_2=\sigma_1(A)$, all examples can be reduced to those where $A$ are diagonal matrices. So, e.g. when $A=\operatorname{diag}(a,b)$ where $a\ge b$, we have $\|A\|_F=\sqrt{a^2+b^2}$ and $\|Av\|_2$ is maximised for a unit vector $v$ when $v=(1,0)^T$. $\endgroup$ – user1551 Nov 13 '18 at 16:51
  • $\begingroup$ Thank you. Also a question on notation: about calling the norm the induced norm, what should I think of as "inducing" the norm? (It shouldn't be $v$ because $v$ is not fixed. So that just leaves $A$ or $Av$?) $\endgroup$ – user106860 Nov 13 '18 at 17:09
  • 1
    $\begingroup$ @user106860 Every vector $p$-norm gives rise to a matrix $p$-norm, so the term means the matrix norm is induced by the vector norm. $\endgroup$ – user1551 Nov 13 '18 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.