Is there an intuitive meaning for the spectral norm of a matrix? Why would an algorithm calculate the relative recovery in spectral norm between two images (i.e. one before the algorithm and the other after)? Thanks


The spectral norm is the maximum singular value of a matrix. Intuitively, you can think of it as the maximum 'scale', by which the matrix can 'stretch' a vector.

  • 1
    $\begingroup$ thanks - this helps me conceptually. $\endgroup$ – val Aug 29 '12 at 6:56
  • 3
    $\begingroup$ This is specifically important since noise will be amplified by this value. For a very nice and intuitive understanding of this I recommend slides 21 and 22 of ee263.stanford.edu/lectures.html . $\endgroup$ – divB May 4 '16 at 7:35
  • $\begingroup$ @divB, of which lecture? $\endgroup$ – gwg Oct 10 '18 at 0:29
  • 1
    $\begingroup$ 27, "SVD and applications" $\endgroup$ – divB Oct 10 '18 at 22:03
  • $\begingroup$ This is only true for symmetric matrices. math.stackexchange.com/questions/1437569/… $\endgroup$ – information_interchange Mar 28 '20 at 2:09

Let us consider the singular value decomposition (SVD) of a matrix $X = U S V^T$, where $U$ and $V$ are matrices containing the left and right singular vectors of $X$ in their columns. $S$ is a diagonal matrix containing the singular values. A intuitive way to think of the norm of $X$ is in terms of the norm of the singular value vector in the diagonal of $S$. This is because the singular values measure the energy of the matrix in various principal directions.

One can now extend the $p$-norm for a finite-dimensional vector to a $m\times n$ matrix by working on this singular value vector:

\begin{align} \|X\|_p &= \left( \sum_{i=1}^{\text{min}(m,n)} \sigma_i^p \right)^{1/p} \end{align}

This is called the Schatten norm of $X$. Specific choices of $p$ yield commonly used matrix norms:

  1. $p=0$: Gives the rank of the matrix (number of non-zero singular values).
  2. $p=1$: Gives the nuclear norm (sum of absolute singular values). This is the tightest convex relaxation of the rank.
  3. $p=2$: Gives the Frobenius norm (square root of the sum of squares of singular values).
  4. $p=\infty$: Gives the spectral norm (max. singular value).
  • 1
    $\begingroup$ thanks - can we really talk about Schatten p=0 for matrices? since we would be looking at 1/0...? $\endgroup$ – val Aug 29 '12 at 7:02
  • $\begingroup$ Yes we can. And it is equal to the rank of the matrix. Can you clarify your question a bit more? $\endgroup$ – Kartik Audhkhasi Aug 29 '12 at 13:41
  • $\begingroup$ Hi - i was referring to the Schatten norm equation above: the exponent is 1/p. If p=0 we have 1/0. thanks.. p must > 1. $\endgroup$ – val Aug 29 '12 at 17:31
  • 3
    $\begingroup$ Think of $p \rightarrow 0$. Then any nonzero singular values will lead to $1$, while $0$ singular values will give $0$ anyway. Hence, you will end up with the number of non-zero singular values. However, for $p \in [0,1]$, the Schatten norm is not a "norm" since it does not satisfy all the properties. However, it is still common practice to call it a "norm". $\endgroup$ – Kartik Audhkhasi Aug 29 '12 at 18:10
  • 2
    $\begingroup$ I just want to point out the confusion in your notation, the same notation: $||A||_2$ is also being used as spectral norm of a Matrix, which is the $p=\infty$ in your answer. I don't know what is Schatten Norm but one thing is universally agreed is that, matrix is an operator, and its norm should be defined in an operator fashion. $\endgroup$ – ArtificiallyIntelligence Nov 15 '18 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.