In my textbook there are few facts left without any sign of a proof, which really bugs me, and I was thinking maybe someone can help me:

  1. $A\in \mathbb{R}^{m,n} \Rightarrow \ \|A\|_2 = \sqrt{\rho(A^TA)}$ where $\rho(A^TA)$ denotes spectral radius of a matrix $A^TA$ i.e. maximum absolute value of an eigenvalue of $A^TA$ (in this case all eigenvalues are real and non-negative)
  2. If $A\in\mathbb{R}^{n,n}$ and $A^T=A$ then $\|A\|_2=\rho(A)$
  3. $\|A\|_2\le\sqrt{\|A^TA\|_{\infty}}$

I suppose these are well known facts, yet I couldn't find their proofs on the Internet. Eigenvalues are very new for me, maybe that is why I couldn't (and why I am not able to prove it alone, I hope they are not too complicated, I prefer simple proofs). In first also fact that eigenvalues of $A^TA$ are real and non-negative is not easy for me.

Additionally I have several questions:

  1. I suppose that (yet I don't know a lot about it) $\|A\|_2=\max_{\vec{x}\neq\vec{0}} \frac{\|A\vec{x}\|_2}{\|\vec{x}\|_2}$ is not easy to compute. Is this second definition of $\|A\|_2$ (with spectral radius) helpful? I mean easier to compute? Can one easy and quite fast find eigenvalues, at least numerically?
  2. Is this inequality in the third point very good, that is $\|A\|_2$ is close to given upper bound?

1 Answer 1

  1. This was asked here already many times: $$ \|A\|_2^2 = \max_{x\neq 0}\frac{\|Ax\|_2^2}{\|x\|_2^2}=\max_{x\neq 0}\frac{x^TA^TAx}{x^Tx}=\lambda_{\max}(A^TA)=\rho(A^TA). $$ The next to last equality is due to the Courant-Fischer theorem.

  2. If $A$ is symmetric, then $\rho(A^TA)=\rho(A^2)=\rho(A)^2$. (Notice that symmetric matrices have real eigenvalues and if a $\lambda$ is an eigenvalue of a symmetric $A$, then $\lambda^2$ is a nonnegative eigenvalue of $A^2$.)

  3. For any (square) $B$ and any matrix norm $\|\cdot\|$ subordinate with respect to some vector norm, we have $\rho(B)\leq\|B\|$. Plug that into the point 1) with the special choice $\|\cdot\|=\|\cdot\|_{\infty}$ to get $\|A\|_2=\sqrt{\rho(A^TA)}\leq\sqrt{\|A^TA\|_{\infty}}$.

Additional 1.

The largest eigenvalue of a matrix is usually quite easy to compute (or estimate) unlike the full spectrum or small eigenvalues but that might (and probably does) depend on the application. There is quite a selection of various numerical methods for computing eigenvalues: the full spectrum or just a part of it. Note that all such algorithms are iterative and may perform well or poorly depending on the matrix.

Additional 2.

IMHO it does not need to be the case generally (sometimes the bound can be tight, sometimes not). However, unlike $\rho(A^TA)$, the $\infty$-norm is almost trivial to compute.

  • $\begingroup$ Thank you very very much! I have only one more question. Why we have for any square $B$ and any matrix norm $\|\cdot\|, \rho(B)\le\|B\|$? Does this also follow from some well known theory? Or is just a simple observation that I don't see? $\endgroup$
    – xan
    Dec 4, 2013 at 10:10
  • 1
    $\begingroup$ @xan It's simple: for any eigenpair $(\lambda,x)$ of $A$, we have from $Ax=\lambda x$ that $\|\lambda x\|=|\lambda|\|x\|=\|Ax\|\leq\|A\|\|x\|$ and hence $|\lambda|\leq\|A\|$. It holds for any vector norm and any matrix norm subordinate w.r.t. that vector norm (that is, a pair of matrix/vector norms satisfying $\|Ax\|\leq\|A\|\|x\|$ for all $x$). In particular, the statement holds for any operator norm such as the $\infty$-norm. $\endgroup$ Dec 4, 2013 at 10:54
  • $\begingroup$ I understand now. Thank you so much! You've helped me a lot. $\endgroup$
    – xan
    Dec 4, 2013 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.