Transform curve into canonical form and determine type I have the curve:
$$9x^2-6xy+y^2+6x-2y-3=0$$
I have to transform it into the nnormal form and determine its type.
Well, we all know that such a curves are given by equation:
$$Ax^2+2Bxy+Cy^2+2Dx+2Ey+F = 0$$
Which is purely the case.

$\textbf{Thoughts:}$
1.) I determined if given curve is central or not:
$$\Delta=\begin{vmatrix} A&B\\B&C \end{vmatrix} = \begin{vmatrix}9&-3\\-3&1 \end{vmatrix} = 9 -9=0$$
Therefore, the curve is not central.

2.) Now have to find an angle of transformation, by the formula:
$$\tan^2(\varphi)+\frac{a_{11}-a_{22}}{a_{12}}\tan(\varphi)-1=0$$
Where:
$$\begin{array}{|c|c|c|}\hline a_{11}&a_{12}&a_{22}\\ \hline 9 & -3 & 1 \\ \hline \end{array}$$
Therefore:
$$\tan^2(\varphi)-\frac{8}{3}\tan(\varphi)-1=0$$
or
$$3\tan^2(\varphi)-8\tan(\varphi)-3=0$$
which is square equation and substitute $t=\tan(\varphi)$ we get:
$$3t^2-8t-1=0$$
And roots are $t_1=3$ and $t_2=-\frac{1}{3}$ (which is strange for tangent, in my opinion).

3.) Find transformation, and appropriate $\sin x$ and $\cos x$
It is well known that there are transformations:
$$\begin{cases} x = x_1\cos(\varphi)-y_1\sin(\varphi) \\ y= x_1\sin(\varphi)+y_1\cos(\varphi) \end{cases} \tag{1}$$
So for $\cos (\varphi)$: $$\cos(\varphi)=\frac{1}{\sqrt{1+\tan^2(\varphi)}}$$
And for $\sin (\varphi)$: $$\sin(\varphi) = \frac{\tan(\varphi)}{\sqrt{1+\tan^2(\varphi)}}$$
As it does not matter which root to apply here I take $3$ and get:
$$\cos(\varphi) = \frac{1}{\sqrt{10}}$$
and
$$\sin(\varphi) = \frac{3}{\sqrt{10}}$$
or, if substitute results into $(1)$ we get:
$$\begin{cases}x = x_1\frac{1}{\sqrt{10}}-y_1\frac{3}{\sqrt{10}} \\ y = x_1\frac{3}{\sqrt{10}}+y_1\frac{1}{\sqrt{10}} \end{cases} \tag{2}$$

4.) System $(2)$ shows us new coordinates, which we should substitute into original equarion, and have:
$$9\left(x_1\frac{1}{\sqrt{10}}-y_1\frac{3}{\sqrt{10}}\right)^2-6\left(x_1\frac{1}{\sqrt{10}}-y_1\frac{3}{\sqrt{10}}\right)\cdot \left(x_1\frac{3}{\sqrt{10}}+y_1\frac{1}{\sqrt{10}} \right) + \left(x_1\frac{3}{\sqrt{10}}+y_1\frac{1}{\sqrt{10}}\right)^2 + 6\left( x_1\frac{1}{\sqrt{10}}-y_1\frac{3}{\sqrt{10}}\right) - 2\left(x_1\frac{3}{\sqrt{10}}+y_1\frac{1}{\sqrt{10}}\right) - 3 = 0 \tag{3}$$
And I am totally stuck here
 A: Hint:
Use Gauß' algorithm to write this polynomial as a sum of squares:
\begin{align}
9x^2-6xy+y^2+6x-2y-3&=(9x^2-6xy+6x)+y^2-2y-3\\
&=\bigl[(3x-y+1)^2-y^2+2y-1\bigr]+y^2-2y-3\\
&=(3x-y+1)^2-4\\
&=(3x-y-1)(3x-y+3).
\end{align}
A: The equation $9x^2-6xy+y^2+6x-2y-3=0$ is equivalent with
$$(x,\; y)\begin{pmatrix} 9 & -3\\ -3 & 1\end{pmatrix}\begin{pmatrix} x\\ y\end{pmatrix}+(6,\;-2)\begin{pmatrix} x\\ y\end{pmatrix}-3=0$$
Now we apply the principal axis theorem: The eigenvalues of the matrix, say $A$, are $\lambda=0$ and $\mu=10$, the eigenvectors $v_{\lambda}=\binom{1}{3}$ and $v_{\mu}=\binom{3}{-1}$. For
$T=\frac{1}{\sqrt{10}}\begin{pmatrix}1&3 \\ 3& -1\end{pmatrix}$ we have $T=T^{-1}=T^{t}$. Consider the transformation $\binom{x}{y}=T\binom{x'}{y'}$, then the above equation is
$$(x',\; y')T^{t}AT\binom{x'}{y'}+(6,\;-2)T\binom{x'}{y'}-3=0$$
which is with $T^{t}AT=\mathrm{diag}(0,10)$ and $(6,\;-2)T=(0,\;20)$ the same as
$$10y'^2+20y'-3=0$$
After a small rearrangement, we get
$$10(y'+1)^2=13$$
For $y''=\frac{y'+1}{\sqrt{13}}$ we arrive at
$$10y''^2=1$$
This equation describes a pair of parallel lines.
