Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How can you prove that: $$\|A\|_2 \le \|A\|_F$$ I cannot use: $$\|A\|_2^2 = \lambda_{max}(A^TA)$$ It makes sense that the 2-norm would be less than or equal to the frobenius norm but I dont know how to prove it. I do know:

$$\|A\|_2 = \max_{\|x\|_2 = 1} {\|Ax\|_2}$$

and I know I can define the frobenius norm to be:

$$\|A\|_F^2 = \sum_{j=1}^n {\|Ae_j\|_2^2}$$ but I dont see how this could help. I dont know how else to compare the two norms though.

share|cite|improve this question
Let $\Vert A\|_2 = \|Av\|$ for some $v$ with $\|v\| = 1$. Define an orthogonal matrix $U$ for which $v$ is the first column. Compute $\Vert AU\Vert_F $ and show that it is equal to $\Vert A\Vert_F $, then compare it to $\Vert Av\Vert$ by a direct computation. – Hans Engler Dec 7 '12 at 4:36
@HansEngler I dont exactly understand your answer. How would I compute $||AU||_2$ and if it is equal to $||A||_2$ then how does that prove anything about $||A||_F$? – user972276 Dec 7 '12 at 4:42
@HansEngler I dont understand how to show $||AU||_F = ||A||_F$. I understand that the first column of AU is just Av and so I can show that $||A||_2 \le ||A||_F$. – user972276 Dec 7 '12 at 4:50
$\Vert A\Vert_F^2 = trace(AA^T)$. Now compute the same thing for $AU$ and use the fact that $U$ is orthogonal. – Hans Engler Dec 7 '12 at 12:14

Write $x=\sum_{j=1}^nc_je_j$, for coefficients $c_1,\ldots,c_n$. Suppose that $\|x\|_2=1$, i.e. $\sum_j |c_j|^2=1$. Then $$ \|Ax\|_2^2=\|\sum_j c_j\,Ae_j\|_2^2\leq\left(\sum_j|c_j|\,\|Ae_j\|_2\right)^{2}\\ \leq\left(\sum_j|c_j|^2\right)\sum_j\|Ae_j\|_2^2=\sum_j\|Ae_j\|_2^2=\|A\|_F^2, $$ where the triangle inequality is used in the first $\leq$ and Cauchy-Schwarz in the second.

As $x$ was arbitrary, we get $\|A\|_2\leq\|A\|_F$.

share|cite|improve this answer
is there a way to do it without using Cauchy-Schwartz? – user972276 Dec 7 '12 at 4:54
Not that I can think of. In any case I simplified the argument by using the triangle inequality (basically, the previous version had a built-in proof of the triangle inequality, which by the way I don't think you can prove without CS). – Martin Argerami Dec 7 '12 at 10:58

In fact, the proof from $\left\| \mathbf{A}\right\|_2 =\max_{\left\| \mathbf{x}\right\|_2=1} \left\| \mathbf{Ax} \right\|_2$ to $\left\| \mathbf{A}\right\|_2 = \sqrt{\lambda_{\max}(\mathbf{A}^H \mathbf{A})}$ is straight forward. We can first simply prove when $\mathbf{P}$ is Hermitian $$ \lambda_{\max} = \max_{\| \mathbf{x} \|_2=1} \mathbf{x}^H \mathbf{Px}. $$ That's because when $\mathbf{P}$ is Hermitian, there exists one and only one unitary matrix $\mathbf{U}$ that can diagonalize $\mathbf{P}$ as $\mathbf{U}^H \mathbf{PU}=\mathbf{D}$ (so $\mathbf{P}=\mathbf{UDU}^H$), where $\mathbf{D}$ is a diagonal matrix with eigenvalues of $\mathbf{P}$ on the diagonal, and the columns of $\mathbf{U}$ are the corresponding eigenvectors. Let $\mathbf{y}=\mathbf{U}^H \mathbf{x}$ and substitute $\mathbf{x} = \mathbf{Uy}$ to the optimization problem, we obtain

$$ \max_{\| \mathbf{x} \|_2=1} \mathbf{x}^H \mathbf{Px} = \max_{\| \mathbf{y} \|_2=1} \mathbf{y}^H \mathbf{Dy} = \max_{\| \mathbf{y} \|_2=1} \sum_{i=1}^n \lambda_i |y_i|^2 \le \lambda_{\max} \max_{\| \mathbf{y} \|_2=1} \sum_{i=1}^n |y_i|^2 = \lambda_{\max} $$

Thus, just by choosing $\mathbf{x}$ as the corresponding eigenvector to the eigenvalue $\lambda_{\max}$, $\max_{\| \mathbf{x} \|_2=1} \mathbf{x}^H \mathbf{Px} = \lambda_{\max}$. This proves $\left\| \mathbf{A}\right\|_2 = \sqrt{\lambda_{\max}(\mathbf{A}^H \mathbf{A})}$.

And then, because the $n\times n$ matrix $\mathbf{A}^H \mathbf{A}$ is positive semidefinite, all of its eigenvalues are not less than zero. Assume $\text{rank}~\mathbf{A}^H \mathbf{A}=r$, we can put the eigenvalues into a decrease order:

$$ \lambda_1 \geq \lambda_2 \geq \lambda_r > \lambda_{r+1} = \cdots = \lambda_n = 0. $$

Because for all $\mathbf{X}\in \mathbb{C}^{n\times n}$, $$ \text{trace}~\mathbf{X} = \sum\limits_{i=1}^{n} \lambda_i, $$ where $\lambda_i$, $i=1,2,\ldots,n$ are eigenvalues of $\mathbf{X}$; and besides, it's easy to verify $$ \left\| \mathbf{A}\right\|_F = \sqrt{\text{trace}~ \mathbf{A}^H \mathbf{A}}. $$

Thus, through $$ \sqrt{\lambda_1} \leq \sqrt{\sum_{i=1}^{n} \lambda_i} \leq \sqrt{r \cdot \lambda_1} $$ we have $$ \left\| \mathbf{A}\right\|_2 \leq \left\| \mathbf{A}\right\|_F \leq \sqrt{r} \left\| \mathbf{A}\right\|_2 $$

share|cite|improve this answer

The Frobenius norm is sub-multiplicative, therefore $||Ax||_F \leq ||A||_F ||x||_F$, which gives:

$$\forall x \neq 0 \quad \frac{ ||Ax||_2 } {||x||_2} = \frac{ ||Ax||_F } {||x||_F} \leq ||A||_F $$

So you have an upper bound for the quotient, and since the supremum (here maximum) is by definition smaller than any other upper bound, you immediately get:

$$ \underset{x \neq 0}{\max}{\frac{||Ax||}{||x||}} =||A||_2 \leq ||A||_F $$

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.