4
$\begingroup$

I have been studying about norms and for a given matrix $A$, I haven't been able to understand the difference between Frobenius norm $\|A\|_F$ and operator $2$-norm $\|A\|_2$. Can someone help me understand the difference between them?

$\endgroup$
2
  • $\begingroup$ Maybe this question is useful. It is not exactly your question, but some answers mention the difference between these norms. $\endgroup$
    – Ernie060
    Commented Nov 13, 2018 at 15:40
  • $\begingroup$ Is that the spectral norm? $\endgroup$ Commented Jul 17, 2023 at 15:14

1 Answer 1

6
$\begingroup$

$\|A\|_F$ is $\|\operatorname{vec}(A)\|_2$, the Euclidean norm of the vector $\operatorname{vec}(A)$ obtained by stacking the columns of $A$ one above the other. So, you just reshape $A$ into a vector and take its Euclidean norm. The Frobenius norm of $A$ can also be expressed as $\sqrt{\operatorname{tr}(A^\ast A)}$, because each diagonal entry of $A^\ast A$ is the squared Euclidean norm of a row of $A$.

$\|A\|_2$, in contrast, is the maximum possible Euclidean norm of $Av$ for a unit vector $v$. Since $A$ a linear operator that operates on $v$, we call it an operator norm. And as we consider the $2$-norm of $Av$, the matrix norm $\|A\|_2$ is also called an induced norm. Had the vector $p$-norm been used in place of the $2$ norm, the resulting matrix norm $\|A\|_p=\max_{\|v\|_p=1}\|Av\|_p$ is called an operator/induced $p$-norm.

For example, $$ \|I_2\|_F=\left\|\pmatrix{1\\ 0\\ 0\\ 1}\right\|_2=\sqrt{2}\ \text{ but } \ \|I_2\|_2=\max_{\|v\|_2=1}\|I_2v\|_2=\max_{\|v\|_2=1}\|v\|_2=1. $$

Both the operator $2$-norm (but not other induced $p$-norms) and Frobenius norm are unitarily invariant, i.e. $\|UAV\|_F=\|A\|_F$ and $\|UAV\|_2=\|A\|_2$ whenever $U,V$ are unitary matrices. Therefore, by singular value decomposition, we always have $$ \|A\|_F = \sqrt{\sum_i\sigma_i(A)^2}\ge\sigma_1(A)=\|A\|_2 $$ and the two matrix norms are equal only when $\sigma_2(A)=\cdots=\sigma_n(A)=0$, i.e. when $\operatorname{rank}(A)\le1$.

The operator $2$-norm is also confusingly called spectral norm in the literature, but as we have seen in the above, $\|A\|_2$ is the largest singular value of $A$. It is not really about the spectrum of $A$. The term probably originated from considering the spectrum of $A^\ast A$.

$\endgroup$
4
  • $\begingroup$ For the induced norm are we taking the maximum over all possible unit vectors $v$ (or is $v$ specified before hand?). If it is the maximum over all possible v, could you perhaps give an example of calculating this when $A$ is not an identity matrix? $\endgroup$
    – user106860
    Commented Nov 13, 2018 at 16:41
  • 1
    $\begingroup$ @user106860 Of course $v$ is not fixed. Since $\|A\|_2=\sigma_1(A)$, all examples can be reduced to those where $A$ are diagonal matrices. So, e.g. when $A=\operatorname{diag}(a,b)$ where $a\ge b$, we have $\|A\|_F=\sqrt{a^2+b^2}$ and $\|Av\|_2$ is maximised for a unit vector $v$ when $v=(1,0)^T$. $\endgroup$
    – user1551
    Commented Nov 13, 2018 at 16:51
  • $\begingroup$ Thank you. Also a question on notation: about calling the norm the induced norm, what should I think of as "inducing" the norm? (It shouldn't be $v$ because $v$ is not fixed. So that just leaves $A$ or $Av$?) $\endgroup$
    – user106860
    Commented Nov 13, 2018 at 17:09
  • 2
    $\begingroup$ @user106860 Every vector $p$-norm gives rise to a matrix $p$-norm, so the term means the matrix norm is induced by the vector norm. $\endgroup$
    – user1551
    Commented Nov 13, 2018 at 18:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .