I'm looking for a clarification of an answer to

Showing that $x^{\top}Ax$ is maximized at $\max \lambda(A)$ for symmetric $A$

(This is a clarified formulation of my original question which I deleted)

The question at the given link asks for a proof that

Given a $n \times n$ symmetric matrix $A$,

$$ \max_{x : ||x||_2 = 1} x^{\top}Ax = \max \lambda(A), $$

where $\max \lambda(A)$ is the maximum eigenvalue of $A$.

I've approached solving the problem in exactly the same way as @Ryan at the above page. The main point is in the upper bound introduced as: $$ x^{\top}Ax = \sum_{i=1}^n \lambda_i \tilde{x}_i^2 \le \max \lambda(A)\sum_{i=1}^n \tilde{x}_i^2 $$

and this is correct.

However, this proves only that $ x^{\top}Ax \le \max \lambda(A),$ but not the original statement $$ \max_{x : ||x||_2 = 1} x^{\top}Ax = \max \lambda(A)$$

  • 3
    $\begingroup$ The usual Lagrange multipliers proof (on the unit sphere) shows that at a maximum point $x$ (which must occur by compactness and continuity), $Ax=\lambda x$ for some constant $\lambda$. This, of course, means that $x$ is an eigenvector with eigenvalue $\lambda$. $\endgroup$ – Ted Shifrin Mar 6 '19 at 22:50
  • $\begingroup$ This exactly clarifies it! Many thanks. Only a true mathematician could answer this, and I'm not the one :) $\endgroup$ – dnqxt Mar 6 '19 at 22:58
  • $\begingroup$ You're welcome. Feel free to check out my various lectures on YouTube (linked in my profile) on multivariable calculus and linear algebra. $\endgroup$ – Ted Shifrin Mar 6 '19 at 23:24

Mazimize $x^TAx$, s.t. $x^Tx=1$. Using Lagrange multiplier $\lambda$ we have $$ \arg\max_{x}( x^TAx - \lambda( x^Tx-1)) $$ so taking derivative w.r.t. vector $x$ and equating to zero we have $$ 2Ax-2\lambda x=0 \to Ax=\lambda x. $$ Namely, the set of solutions of this maximization problem should satisfy $ Ax = \lambda x$, i.e., such vectors $x$s are vectors that multiplying $A$ by $x$ is the same as multiplying $x$ by a scalar. That is, by definition $x$ have to be the eigenvectors and thus $\lambda$ its corresponding eigenvalue. Now, you have a set of solutions $\{(\lambda_i, x_i)\}_{i=1}^n$ and you have to choose the pair the maximize the original function, so clearly $\max x^TAx = x ^T \lambda_{\max} x = \lambda_{\max} x^T x = \lambda_{\max}.$


Are you sure about this statement "then realized that only the strict inequality is justified in front of the upper bound."?

Take for example $A=I$

  • $\begingroup$ Thank you for your answer. I clarified the question, could you please respond to it. $\endgroup$ – dnqxt Mar 6 '19 at 22:41
  • 1
    $\begingroup$ I second the answer above ! $\endgroup$ – Pebeto Mar 6 '19 at 23:03

If $A$ is symmetric, then its eigenvalues are real. Moreover, we have (Rayleigh coefficient) $$ \frac{x^TAx}{||x||^2} = \lambda $$ and since $||x||^2=1$, we get $x^T A x = \lambda$. So, $$ \max_{||x||^2 = 1} x^T A x = \max \lambda. $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.