# Meaning of the spectral norm of a matrix

Is there an intuitive meaning for the spectral norm of a matrix? Why would an algorithm calculate the relative recovery in spectral norm between two images (i.e. one before the algorithm and the other after)? Thanks

## 2 Answers

The spectral norm is the maximum singular value of a matrix. Intuitively, you can think of it as the maximum 'scale', by which the matrix can 'stretch' a vector.

• thanks - this helps me conceptually. – val Aug 29 '12 at 6:56
• This is specifically important since noise will be amplified by this value. For a very nice and intuitive understanding of this I recommend slides 21 and 22 of ee263.stanford.edu/lectures.html . – divB May 4 '16 at 7:35
• @divB, of which lecture? – gwg Oct 10 '18 at 0:29
• 27, "SVD and applications" – divB Oct 10 '18 at 22:03
• This is only true for symmetric matrices. math.stackexchange.com/questions/1437569/… – information_interchange Mar 28 '20 at 2:09

Let us consider the singular value decomposition (SVD) of a matrix $$X = U S V^T$$, where $$U$$ and $$V$$ are matrices containing the left and right singular vectors of $$X$$ in their columns. $$S$$ is a diagonal matrix containing the singular values. A intuitive way to think of the norm of $$X$$ is in terms of the norm of the singular value vector in the diagonal of $$S$$. This is because the singular values measure the energy of the matrix in various principal directions.

One can now extend the $$p$$-norm for a finite-dimensional vector to a $$m\times n$$ matrix by working on this singular value vector:

\begin{align} \|X\|_p &= \left( \sum_{i=1}^{\text{min}(m,n)} \sigma_i^p \right)^{1/p} \end{align}

This is called the Schatten norm of $$X$$. Specific choices of $$p$$ yield commonly used matrix norms:

1. $$p=0$$: Gives the rank of the matrix (number of non-zero singular values).
2. $$p=1$$: Gives the nuclear norm (sum of absolute singular values). This is the tightest convex relaxation of the rank.
3. $$p=2$$: Gives the Frobenius norm (square root of the sum of squares of singular values).
4. $$p=\infty$$: Gives the spectral norm (max. singular value).
• thanks - can we really talk about Schatten p=0 for matrices? since we would be looking at 1/0...? – val Aug 29 '12 at 7:02
• Yes we can. And it is equal to the rank of the matrix. Can you clarify your question a bit more? – Kartik Audhkhasi Aug 29 '12 at 13:41
• Hi - i was referring to the Schatten norm equation above: the exponent is 1/p. If p=0 we have 1/0. thanks.. p must > 1. – val Aug 29 '12 at 17:31
• Think of $p \rightarrow 0$. Then any nonzero singular values will lead to $1$, while $0$ singular values will give $0$ anyway. Hence, you will end up with the number of non-zero singular values. However, for $p \in [0,1]$, the Schatten norm is not a "norm" since it does not satisfy all the properties. However, it is still common practice to call it a "norm". – Kartik Audhkhasi Aug 29 '12 at 18:10
• I just want to point out the confusion in your notation, the same notation: $||A||_2$ is also being used as spectral norm of a Matrix, which is the $p=\infty$ in your answer. I don't know what is Schatten Norm but one thing is universally agreed is that, matrix is an operator, and its norm should be defined in an operator fashion. – ArtificiallyIntelligence Nov 15 '18 at 19:26