intersection of a plane and a hyperbola

I have done the first bit, I have changed basis and got the eigenvalues and eigenvectors:

$$\mathbf{r^T} \begin{bmatrix} 1/\sqrt{2} & -1/3\sqrt{2} & -2/3 \\ 0 & -4/3\sqrt{2} & 1/3 \\ -1/\sqrt{2} & 1/3\sqrt{2} & 2/3 \end{bmatrix}^T \begin{bmatrix} -1 & 0 & 0 \\ 0 & -5 & 0 \\ 0 & 0 & 4 \end{bmatrix} \begin{bmatrix} 1/\sqrt{2} & -1/3\sqrt{2} & -2/3 \\ 0 & -4/3\sqrt{2} & 1/3 \\ -1/\sqrt{2} & 1/3\sqrt{2} & 2/3 \end{bmatrix} \mathbf{r}$$

So I have the Hyperbola: $-X^2-5Y^2+4Z^2=5$ I don't really understand hyperbolas, what do the -1 and -5 correspond to. Also, I tried changing the plane into the new basis and it didn't help, I was hoping it would be along one of the axis (in fact it was meant to be, they made a mistake but said it should still be doable).

Any help in continuing the question would be helpful.

• To begin with, check your work so far. One of the vectors that you’ve got isn’t really an eigenvector of $Q_1$. Second, that big matrix product that you’ve written down wouldn’t equal $Q_1$ even if you had gotten the correct eigenvector. It appears that you’ve gotten the change of basis backwards. – amd Jun 1 '17 at 8:08
• In the future, please take the time to type in critical content instead of simply pasting a picture of it. Images are neither searchable nor accessible to people using screen readers. – amd Jun 1 '17 at 8:34
• I'll remember the image advice, thank you. I also didn't realise the order of basis change mattered so thanks for that as well. – Toby Peterken Jun 2 '17 at 13:35

Let’s get a few things out of the way before plunging into the interesting part. First, having to simultaneously diagonalize $Q_2$ is something of a red herring. Its matrix in the standard basis is the identity matrix, and will be so in any orthonormal basis. This reflects the symmetry of the sphere represented by $Q_2$.
Second, it appears that you’ve made a sign error in computing the eigenbasis of $Q_1$: $$Q_1\left(\frac1{\sqrt2},0,-\frac1{\sqrt2}\right)^T=\left(\frac3{\sqrt2},-2\sqrt2,-\frac3{\sqrt2}\right)^T$$ which is obviously not a multiple of $\left(\frac1{\sqrt2},0,-\frac1{\sqrt2}\right)^T$. On the other hand, $\left(\frac1{\sqrt2},0,\frac1{\sqrt2}\right)^T$ is an eigenvector of $-1$. You’ve also gotten a sign wrong in the transformed equation of the surface. It should be $-X^2-5Y^2+4Z^2=-5$. You need to be especially careful about this when working with quadratic forms since a single change of sign in the signature can completely change the nature of the resulting surface.
Third, you’ve gotten the change of basis backwards. If you actually multiply out the matrix product that you’ve written down, you don’t end up with $Q_1$ (or any scalar multiple of it). The matrix $U$ of normalized eigenvectors maps from the eigenbasis to the standard basis, so $Q_1=(U^{-1})^T\Lambda(U^{-1})=U\Lambda U^T$, not $U^T\Lambda U$ as you have it.
All that aside, you’ve computed the eigenvalues of $Q_1$ correctly. From these eigenvalues, we see that the quadratic form has a signature of $--+$, so the surface $\mathbf r^TQ_1\mathbf r=k$ is an elliptic hyperboloid of one sheet for $k\lt0$ and a hyperboloid of two sheets for $k\gt0$, centered on the origin. Had the plane in the problem been aligned with the principal axes of $Q_1$, i.e., had a normal of $(-2,1,2)^T$ instead of $(2,1,-2)^T$, describing the intersection of the quadric surface with the plane would’ve been simple: that normal corresponds to the $Z$-axis, so simply set $Z=0$ in the equation of the surface to get the ellipse $X^2+5Y^2=5$. (This typo in the problem kind of defeats the purpose of the exercise, I think.) In fact, the intersection of every plane orthogonal to the $Z$-axis with the surface is an ellipse. To complete the description of the surface and sketch it, examine the cross-sections that are obtained by holding $X$ or $Y$ constant instead of $Z$.
The intersection of the surface with plane actually given in the problem isn’t too hard to compute, either, but it does require a bit more work. We’ll work with a slightly different eigenbasis than yours—one with the second vector negated so that it has the same orientation as the standard basis. The change-of-basis matrix is then $$U=\pmatrix{\frac1{\sqrt2}&\frac1{3\sqrt2}&-\frac23\\0&\frac4{3\sqrt2}&\frac13\\\frac1{\sqrt2}&-\frac1{3\sqrt2}&\frac23}.$$ The first order of business is to find an orthonormal basis for the plane. We’ll work in the eigenbasis to make the later matrix multiplications simpler. The normal vector to the plane is given as $\mathbf n=(2,1,-2)^T$, which in our eigenbasis is $\mathbf n'=U^T\mathbf n=\left(0,\frac{4\sqrt2}3,-\frac73\right)^T$. Obviously, $\mathbf u'=(1,0,0)^T$ is orthogonal to this, which is fortunate because this means that the resulting curve will have the $X$-axis as one of its principal axes. For the other basis vector we take $\mathbf n'\times\mathbf u'$, normalized: $\mathbf v'=\left(0,-\frac79,-\frac{4\sqrt2}9\right)$. Every vector $\mathbf r'$ in the plane can be expressed as a linear combination $\mathbf r'=\lambda\mathbf u'+\mu\mathbf v'$ of these two vectors. We then have $$\lambda\mathbf u'+\mu\mathbf v'=\left(\lambda,-\frac79\mu,-\frac{4\sqrt2}9\mu\right)^T$$ and \begin{align}\pmatrix{\lambda & -\frac79\mu & -\frac{4\sqrt2}9\mu}\pmatrix{-1&0&0\\0&-5&0\\0&0&4}\pmatrix{\lambda \\ -\frac79\mu \\ -\frac{4\sqrt2}9\mu}&=\pmatrix{\lambda & -\frac79\mu & -\frac{4\sqrt2}9\mu}\pmatrix{-\lambda \\ \frac{35}9\mu \\ -\frac{16\sqrt2}9\mu}\\&=-\lambda^2-\frac{13}9\mu^2=-5.\end{align} This is the equation of an ellipse aligned with $\mathbf u'$ and $\mathbf v'$, with semi-axis lengths $\sqrt5$ and $3\sqrt{\frac5{13}}$, respectively.
By the way, in the very first part of the problem, we know without computing the entire characteristic polynomial what $c$ must be from Vieta’s formulas and the properties of the trace of a matrix: $c=-\operatorname{tr}Q_1$.