Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to prove the sentence "A short calculation shows the above leads to M u = λ u (symmetry of M is needed here). Therefore λ is the largest eigenvalue of M." from this wikipedia site about Singular Value Decomposition. Shortly, show $$\nabla \bar{x}^{T} M \bar{x} = \lambda ( \nabla \bar{x}^{T} \bar{x} )$$

satisfying $M \bar{u} = \lambda \bar{u}$ where $\bar{u}$ is the maximum eigen vector, $\bar{M}$ is a symmetric matrix and $\lambda$ is eigen value, corresponding to the maximum eigen vector.


Maximize $\bar{x}^{T} M \bar{x}$ over all $\bar{x}$ such that $||\bar{x}||=1$. Lagrange function is $$f(x) = \bar{x}^{T} M \bar{x} + \mu (||\bar{x}||- 1)$$

Now I need to calculate $$\nabla f(x)$$

calculating, and trying to show that the largest eigen value occur with the constraint...calculating and thinking.

Nabla operations here.

share|cite|improve this question
Is is not $(\lambda\cdot \nabla)x^Tx$, but $\lambda(\nabla x^Tx)$ -- just a scalar multiple of the gradient. – Henning Makholm Sep 8 '11 at 17:48
Henning Makholm: thank you, yes that changes the equation totally, have to reconsider it now, fixed it. – hhh Sep 8 '11 at 17:55
Also, the correct terminology for $Mu=\lambda u$ is that $u$ is an eigen_vector_, whereas $\lambda$ is its eigen_value_. – Henning Makholm Sep 8 '11 at 17:57
Henning Makholm: you mean Wikipedia has a mistake with the dot-product here, the point existence there. – hhh Sep 8 '11 at 18:02
Using a centered dot for scalar multiples (or even for multiplication of two numbers) is not wrong, so I wouldn't say that... – Henning Makholm Sep 8 '11 at 18:06
up vote 1 down vote accepted

I'm not sure we agree on the interpretation of the Wikipedia paragraph. What it's stated there (as I see it) is that, starting from the equation

$$\nabla \bar{x}^{T} M \bar{x} = \lambda ( \nabla \bar{x}^{T} \bar{x} )$$

we obtaing that (if M is symmetric) $\lambda$ must be an eigenvalue - and hence $x$ and eigenvector. This is true, and it's rather easy from these two (useful) gradients:

$$\nabla (x^T x) = 2 x $$ $$\nabla (x^T M x) = 2 M x $$

These are the vectorial generalization of $\frac{\partial (x^2)}{\partial x} = 2x $ and $\frac{\partial ( m x^2) }{\partial x} = 2 m x $, and can be proved by definition.

Now, the first equation was obtained by applying Lagrange multipliers to the problem of maximizing $x^T M x$ subject to $x^T x = 1$. Specifically: we write

$$\nabla [x^T M x + \mu(x^Tx - 1)]=0 \Rightarrow \nabla( {x}^{T} M {x}) = \lambda \nabla ({x}^{T} {x} ) \hspace{40px} (\lambda = -\mu)$$

We get then that $\lambda$ (the Lagrange multiplier) must be $some$ eigenvalue, and $x$ some eigenvector. But we want to maximize $x^T M x $, so we must choose the eigenvector with largest eigenvalue, because $x^T M x = x^T \lambda x = \lambda$

share|cite|improve this answer
is there one minus missing in the point of implication? $\nabla x^{T} Mx = -\lambda (\nabla x^{T} x)$. – hhh Sep 8 '11 at 19:04
In which points, you are using the formulas $\nabla(x^{T} x) = 2x$ and $\nabla(x^{T}Mx) = 2Mx$? Is it correct below? $$\nabla(x^{T}Mx) = 2Mx = 2x M = \nabla (x^{T} x) M$$ – hhh Sep 8 '11 at 19:08
@hhh fixed, thanks – leonbloy Sep 8 '11 at 19:10
@hhh : Multiply like this $\begin{pmatrix}-&-\end{pmatrix} \begin{pmatrix}\cdot&-&\cdot\\| &\ddots &|\\\cdot&-&\cdot\end{pmatrix}\begin{pmatrix}|\\|\end{pmatrix}$. Can you see it? Btw, I love ASCII art :) – user13838 Sep 8 '11 at 19:34
now I see it, of course. And the $Q^{T}\sum Q$ is just a general case, returning matrix with scalars. $Q$ has vectors such as the $x$. Thanks. @percusse – hhh Sep 9 '11 at 16:39

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.