Pardon if this question is a bit uninspiring, but the few resources I've found seem to have different definitions. Can someone please provide a definition of the $L_1$, $L_2$, $L_{\infty}$, and $L_p$ norms for Matrices? If you are writing a formal definition would you please also explain it in laymans terms as well; thanks.

  • $\begingroup$ The $L^p$ norms for matrices are the same as they would be for vectors. In fact with these norms, you effectively treat the matrices as vectors by concatenating the columns (or rows..). $\endgroup$ – Cameron Williams Mar 12 '19 at 19:08
  • 2
    $\begingroup$ These are called the induced norms and you can find more about them here. There are equivalent ways of defining many of them, but they correspond to how much the matrix seen as an operator can inflate vectors (with respect to the $L_p$ norm you wish). $\endgroup$ – ippiki-ookami Mar 12 '19 at 19:09
  • 1
    $\begingroup$ @CameronWilliams Usually the $L_p$-norm of a matrix refers to the matrix norm induced by the corresponding vector $L_p$-norm, which is different than just treating a matrix as a vector. $\endgroup$ – littleO Mar 12 '19 at 19:42

Let $p \geq 1$. The $p$-norm of a real $m \times n$ matrix $A$ is the matrix norm induced by the vector $p$-norm. In other words: $$ \|A\|_p = \sup \{ \|Ax \|_p \mid \| x \|_p =1\}. $$ It tells you how much the $p$-norm of a vector can be amplified when you multiply by $A$.

You can show that $\| A \|_1$ is the maximum absolute column sum of $A$, $\| A \|_\infty$ is the maximum absolute row sum of $A$, and $\| A \|_2$ is the largest singular value of $A$.

Further comments: Here is a brief explanation or intuition for the above facts. To see that $\| A \|_1$ is the maximum absolute column sum, we can think about how to select $x$ so that $\| Ax \|_1$ is as large as possible. In selecting the components of $x$, we have a "budget" constraint that $\| x \|_1 = 1$. $Ax$ is a linear combination of the columns of $A$, and it makes sense to spend the entire budget on the best column of $A$.

To make $\| Ax \|_\infty$ as large as possible, subject to the constraint that $\|x\|_\infty = 1$, note that $Ax$ has one component for each row of $A$. The best choice of $x$ will have all components equal to $\pm 1$.

If $A = U \Sigma V^T$ is the SVD of $A$, then $\| Ax \|_2 = \| U \Sigma V^T x \|_2 = \| \Sigma y \|_2$, where $y = V^T x$. To make $\|Ax\|_2$ as large as possible, subject to $\|x\|_2 = \|y\|_2 = 1$, we should arrange for the entire mass of $y$ to be concentrated in one component, corresponding to the largest diagonal element of $\Sigma$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.