Why are the axes of a quadratic form ellipse along the eigenvectors? Suppose $A$ is a $2$ x $2$ real symmetric matrix, then
$$x'Ax =\lambda_1(e_1'x)'(e_1'x) + \lambda_2(e_2'x)'(e_2'x) = c^2$$
where $x$ is an arbitrary $2$ x $1$ vector, $e_i$ and $\lambda_i$ the eigenvectors and eigenvalues of $A$, and $c$ is a constant. If we set $y_i = e_i'x$ then the above becomes 
$$x'Ax =\lambda_1y_1'y_1 + \lambda_2y_2'y_2 = c^2$$
$$x'Ax =\lambda_1y_1^2 + \lambda_2y_2^2 = c^2$$
And this defines the ellipse
$$\bigg(\frac{y_1}{c/\sqrt\lambda_1}\bigg)^2 + \bigg(\frac{y_2}{c/\sqrt\lambda_2}\bigg)^2 = 1$$
It's said that this ellipse has axes along the eigenvectors of $A$, because $y_i = e_i'x$. But can't this also be an ellipse along $x$? Also the eigenvectors have more than one component but the $y_i$'s are a scalar, how do you get direction from a scalar?
 A: If $y_1$ and $y_2$ are scalar variables,
in order for an equation such as
$$
\left(\frac{y_1}{c/\sqrt\lambda_1}\right)^2 + \left(\frac{y_2}{c/\sqrt\lambda_2}\right)^2 = 1  \tag1
$$
to define an ellipse these two scalars $y_1$ and $y_2$ must somehow identify a point in the plane. How can two scalars identify a point? 
In this case, $y_1$ and $y_2$ identify a point by specifying a linear combination of two basis vectors.
Specifically, by setting $y_i = e_i' x,$ we set $e_1$ and $e_2$ as the basis vectors for $y_1$ and $y_2.$  We thereby ensure that
$$ x = y_1 e_1 + y_2 e_2 .$$
(This assumes $e_i$ are unit vectors; otherwise your formulas do not work.)
The direction of the axes of the ellipse is the direction of the basis vectors,
because that is the kind of ellipse that an equation of the form of Equation $1$ can describe.
An ellipse with an axis not aligned with the basis of the coordinate system
would have an additional term, a non-zero multiple of $y_1y_2.$
Another way to put it is, the equation of any ellipse whose center is at the origin can be written in the form $x'Ax = c^2.$
Once you have written the equation for a particular ellipse, the equation describes that ellipse, not any other ellipse.
Unless the equation describes a circle,
the ellipse described by the equation has a major axis in one particular direction, which may or may not coincide with an axis of a particular coordinate system or with any other particular vector you might consider.
In general, its equation in a given coordinate system cannot be written in the form of Equation $1$
because the form $x'Ax,$ written in terms of individual coordinates, includes as one of its terms a multiple of $x_1x_2$ that is not necessarily zero.
In order to get an equation like Equation $1,$ we can orthogonally transform (rotate and/or reflect) the coordinate system in order to have coordinate axes that align with the axes of the ellipse.
And the correct choice of coordinate axes in order to get an equation
like Equation $1$ are the axes aligned with the eigenvectors of $A.$
As for an "ellipse along $x,$" what would that be?
When we say the quadratic form $x'Ax = c^2$ defines an ellipse,
we take $x$ as a variable vector; in order to plot all the points of the ellipse, we need to include vectors in every direction among our solution set.
So of course the (major) axis of the ellipse is along one value of $x$
among all values in the solution set, but so is the minor axis, and so is
every other line through the center of the ellipse.

Note that simply knowing that $A$ is symmetric does not tell you that
$x'Ax=c^2$ is the equation of an ellipse.
If $A=\begin{pmatrix} 1&0 \\ 0&-1 \end{pmatrix}$ and $c=1,$ for example, the equation describes a hyperbola; if $A=\begin{pmatrix} 1&0 \\ 0&-1 \end{pmatrix}$ and $c\neq 0$ it is two intersecting lines;
if $A=\begin{pmatrix} 1&0 \\ 0&0 \end{pmatrix}$ and $c=1,$ two parallel lines;
if $A=\begin{pmatrix} 1&0 \\ 0&0 \end{pmatrix}$ and $c=0,$ one line.
Equation $1$ describes an ellipse only when both eigenvalues of $A$ are positive, because we need $c/\sqrt{\lambda_i}$ to be real and non-zero.
