# Trouble with geometrical application of Lagrange multiplier

Let $A\in\mathbb R^{n\times n}$ be positive-definite and $\langle Ax,x\rangle=1$ be the equation of an ellipsoid $M\subset\mathbb R^n$. Use Lagrange multipliers to prove that the greatest distance of a point in $M$ to the origin is the smallest eigenvalue of $A$ raised to $-1/2$.

$A$ is too general, so I tried using that there is an invertible matrix $P$ such that $D\triangleq PAP^{-1}$ is diagonal to solve the problem with $D$ and then go back to the original problem. For simplicity, I'm maximizing the squared distance, so, with $\tilde x$ being the critical distance, the system of equations is \left\{\begin{align} \nabla\|x\|^2&=\lambda\nabla\langle Dx,x\rangle\Leftarrow x=\tilde x\\ \langle D\tilde x,\tilde x\rangle&=1 \end{align}\right.\quad. This is equivalent to \left\{\begin{align} \nabla\left(\sum_{i=1}^n x_i^2\right)&=\lambda\nabla\left(\sum_{i=1}^n\mu_i x_i^2\right)\Leftarrow x=\tilde x\\ \sum_{i=1}^n\mu_i\tilde x_i^2&=1 \end{align}\right.\quad, where $\mu_1,\ldots,\mu_n$ are the diagonal entries of $D$, which are the eigenvalues of $A$. The first equation gives $$2\tilde x_i=2\lambda\mu_i\tilde x_i : i=1,\ldots,n\quad.$$ If every $\tilde x_i$ is zero, the restriction is impossible, because every $\mu_i$ is positive, thus I get the very questionable conclusion that every $\mu_i$ equals $1/\lambda$. What is going on?

-
Since the $\tilde{x}_i$ aren't linearly independent on $M$, you only get that $$\sum_{i=1}^n (1-\lambda \mu_i)x_i = 0.$$ –  Branimir Ćaćić Apr 22 '13 at 19:05

Let $f(x)=\langle Ax,x\rangle$. The gradient is better computed as a directional derivative: $$\langle\nabla f(x),u\rangle=\lim_{t\to 0}\frac{f(x+tu)-f(x)}{t}=\cdots=\langle Ax,u\rangle+\langle Au,x\rangle=\langle 2Ax,u\rangle,$$ where the dots are straightforward and the last equality follows because $A$ is symmetric. Thus, $\nabla f(x)=2Ax$. Now the function to maximize is $h(x)=\langle x,x\rangle$ with $\nabla h(x)=2x$ (as above with $A=I$). Now the multiplier is some $\lambda$ with $\nabla h(x)=\lambda\nabla f(x)$. In our case $2x=2\lambda Ax$, so that $x$ is an eigenvector with eigenvalue $\mu=1/\lambda$ ($x=0$ is not in the ellipsoid, hence $\lambda\ne0$). Furthermore $$h(x)=\langle x,x\rangle=\langle\lambda Ax,x\rangle=\lambda\langle Ax,x\rangle=\lambda.$$ Thus $h(x)$ is maximum for $\lambda$ maximum, hence $\mu$ minimum. We conclude that the maximum distance is $\sqrt{\lambda}=\sqrt{1/\mu}=\mu^{-1/2}$, where $\mu$ is the minimum eigenvalue of $A$. Note that $A$ being positive definite, its eigenvalues are all $>0$ and the square root does exist.
This is just the way to write in general the argument you have in the diagonal case. On the other hand, the result follows from that particular case, because by the Spectral Thm, there is a change of variables as you write $D=PAP^{-1}$ that is an isometry ($P$ is orthogonal: $P^{-1}=P^T$). Since isometries preserve distances and eigenvalues, what works for $D$, works for $A$.
Geometrically, once you have the euclidean diagonal form, the maximum distance is given by the biggest axis, which is of course the inverse of the smallest eigenvalue. (Think of the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$.)