# Help understanding the geometry behind the maximum principle for eigenvalues.

Context:

I am learning about the maximum principle, which states that

If $$A$$ is a real symmetric matrix and $$q(x)=\langle Ax, x \rangle$$, the following statements hold:

1. $$\lambda_1=\mathrm{max}_{||x||=1} q(x)=q(x_1)$$ is the largest eigenvalue of the matrix $$A$$ and $$x_1$$ is the eigenvector corresponding to eigenvalue $$\lambda_1$$.
2. Let $$\lambda_k=\mathrm{max} q(x)$$ subject to the constraints $$\langle x,x_j \rangle=0 \ \ \ j=1,2,\dots ,k-1.$$ $$||x||=1$$.

Then $$\lambda_k=q(x_k)$$ is the $$kth$$ eigenvalue of $$A$$, $$\lambda_1\geq \lambda_2 \geq \dots \geq \lambda_k$$ and $$x_k$$ is the corresponding eigenvector of $$A$$.

Now the book states that geometrically, we can visualize this as follows:

To understand the geometric significance of the maximum principle, we view the quadratic form $$q(x)=\langle Ax,x \rangle$$ as a radial map of the sphere. That is, to each point on the sphere $$||x||=1$$, associate $$q(x)$$ with the projection of $$Ax$$ onto $$x$$, $$\langle Ax,x \rangle x$$.If $$A$$ is positive definite, the surface $$q(x)x$$ is an ellipsoid. The maximum of $$q(x)$$ occurs exactly along the major axis of the ellipsoid. If we intersect it with the plane orthogonal to the major axis, then restricted to this plane, the maximum of q(x) occurs along the semi-major axis and so on.

I've bolded the parts that I don't understand. I don't see how this radial mapping is transforming the sphere into an ellipsoid nor the parts about the maximums occuring along the major axis and "so on". Could anyone elucidate this geometric idea for me?

The term 'radial map' just means that a vector $x$ is mapped to a parallel vector, $cx$, $c \in \mathbb{R}$. This means that $x$ has only moved in the radial direction to get to its image. The function $q(x)$ is real-valued, but $q(x)x$ is a rescaling of the vector $x$ by a factor $\langle Ax,x \rangle$, which is the length of the projection of $Ax$ onto the unit vector $x$.

To see that this map will turn the sphere $\{x : \|x\| = 1 \}$ into an ellipsoid, consider the transformation $A = \left[ \begin{array}{cc} 3 & 1 \\ 1 & 3 \end{array} \right]$ on $\mathbb{R}^2$. A point on the unit circle looks like $(\cos \theta, \sin \theta)$ for some $\theta$; this is sent to the point $(3\cos\theta + \sin\theta, \cos\theta + 3\sin\theta)$, which lies (you can check) on the ellipse $\left({x-y \over 2} \right)^2 + \left({x + y \over 4} \right)^2 = 2$. This can be calculated by solving the eigenvalue problem for the matrix, and your theorem says we'll always get a similar result (due to the complete set of orthogonal eigenvectors). Anyway, you get essentially the same picture in higher dimensions, but the resulting shape is an ellipsoid. The ellipse above looks like a circle that has been stretched by a factor of 4 in the $x$-direction and a factor of $2$ in the $y$-direction, then rotated $\pi/4$ radians counterclockwise. This puts the major axis (longest diameter across the ellipse, in blue) along the line $y = x$.

Since this is the longest diameter, it is the point where $q(x)x$ has 'stretched the most,' so this is where $q(x)$ is largest. Hence the book's statement that the "maximum of $q(x)$ occurs exactly along the major axis of the ellipsoid." Since $q(x)x$ is a radial map, this shows that the vector $(\cos(\pi/4),\sin(\pi/4)) = (1/\sqrt{2},1/\sqrt{2})$ (which gets mapped to the tip of the blue line) is the eigenvector corresponding to the largest eigenvalue.

The eigenvectors are orthogonal, so to find the eigenvector corresponding to the next largest eigenvalue we 'delete' the first eigenspace we've found. This is done by projecting onto its orthogonal complement. In our example, the orthogonal complement in $\mathbb{R}^2$ of $y = x$ is the line $y = -x$ (the green minor axis). The two points where the ellipse intersects this subspace are $\pm(\sqrt{2},-\sqrt{2})$. This (one-dimensional, trivial) ellipse has major axis along $y = -x$ and the vector in that direction, $(1/\sqrt{2}, -1/\sqrt{2})$, is stretched by a factor of $2$. So the second eigenvector points in the $(1,-1)$ direction and has eigenvalue $2$.

If we had started with a 3D ellipsoid, projecting along the major axis would leave us with a 2D ellipse with its own major axis. The theorem says we can always find a major axis and the other eigenspaces will be orthogonal to it, allowing us to note the axis' direction and length, then project and find the next vector-value pair.

The eigenvectors for a real symmetric matrix are a complete orthonormal set, that is we can take them a an orthonormal basis for $R^n$.So we can represent any $v$ on the unit $n$-sphere as $(v_j)_{1\le j \le n}$ in the sense that there is a unique $(v_j)_{1 \le j \le n}$ such that $v=\sum_j (v_j x_j)$, and furthermore we have $\sum_j (v_j)^2=1$. That is, the lines from the origin ,through the eigenvectors, form a system of mutually orthogonal co-ordinate axes. And $Av= \sum_j (\lambda_j v_j x_j).$