Equation of Conics The general equation of the conic section is : $ax^2+2hxy+by^2+2gx+2fy+c=0$,
where
$$\Delta=\begin{vmatrix}a&h&g\\h&b&f \\g&f &c\\
\end{vmatrix}$$
This equation can also be analysed to distinguish whether it is an equation of pair of straight lines, parabola, ellipse or hyperbola.

*

*If $\Delta=0$ and $h^2=ab$, it represents pair of straight lines

*If $\Delta \neq 0$ and $h^2=ab$, it represents a parabola

*If $\Delta \neq 0$ and $h^2<ab$, it represents an ellipse

*If $\Delta\neq 0$ and $h^2>ab$, it represents a hyperbola

So my question is
Can you represent a proof for why these are the relations for different shapes?
I would prefer a more analytic/geometric approach
 A: This answer will rely on the fact that any plane section of a right elliptical cone is also a plane section of a right circular cone.
This fact can be justified by applying an affine transformation that dilates the cone only in the direction of the minor axis of the elliptical base in order to transform the base to a circle, provided that it has also been shown that the affine transformation of a conic section is a conic section of the same type.
(Also see the question, What if we have a cone with elliptical base?.)
I will also classify a circle as a special case of an ellipse so that I can write simply "an ellipse" rather than "an ellipse or a circle".

Observe that if
$$ M = \begin{pmatrix} a&h&g \\ h&b&f \\ g&f&c \end{pmatrix}
\qquad \text{and} \qquad  \mathbf x = \begin{pmatrix} x\\y\\z \end{pmatrix} $$
then $\Delta$ is the determinant of $M$.
Moreover, $\mathbf x^\top M \mathbf x$
(where the superscript $^\top$ denotes the transpose)
is a homogeneous quadratic polynomial,
\begin{align}
\mathbf x^\top M \mathbf x
&= \begin{pmatrix} x&y&z \end{pmatrix}
   \begin{pmatrix} a&h&g \\ h&b&f \\ g&f&c \end{pmatrix}
   \begin{pmatrix} x\\y\\z \end{pmatrix} \\
&= ax^2+2hxy+by^2+2gxz+2fyz+cz^2.
\end{align}
If we simultaneously set $\mathbf x^\top M \mathbf x = 0$ and $z = 1,$
we get the equation
$$ax^2+2hxy+by^2+2gx+2fy+c = 0, \tag1 $$
but we also get the intersection of the three-dimensional figure that satisfies the equation
$$ \mathbf x^\top M \mathbf x = 0 \tag2 $$
with the plane $z = 1.$
A symmetric real matrix $M$ can always be diagonalized,
that is, it can be expressed as a product of the form
$R^\top DR$ where $R$ is an orthogonal matrix and $D$ is a diagonal matrix.
So Equation $(2)$ can be rewritten
$$ 0 = \mathbf x^\top R^\top DR \mathbf x = (R\mathbf x)^\top D (R \mathbf x). \tag3 $$
Observe that $R\mathbf x$ is just a rotation of the vector $\mathbf x$.
Let the coordinates of $R\mathbf x$ and the entries of $D$ be
$$
  R\mathbf x = \begin{pmatrix} x'\\y'\\z' \end{pmatrix}
\qquad \text{and} \qquad 
 D = \begin{pmatrix} a'&0&0 \\ 0&b'&0 \\ 0&0&c' \end{pmatrix}.
$$
Equation $(3)$ then is equivalent to
$$ a' x'^2 + b' y'^2 + c' z'^2 = 0. \tag4 $$

Note that $\Delta \neq 0$ if and only $M$ is invertible, which is true if and only if
$a',$ $b',$ and $c'$ are all non-zero.
Let's consider first the case $\Delta \neq 0$.
If $a',$ $b',$ and $c'$ are all non-zero and
all have the same sign (all positive or all negative)
then the only solution of Equation $(4)$ is $x' = y' = z' = 0,$ that is,
$R\mathbf x = 0.$
But because $R\mathbf x$ is a rotation of $\mathbf x,$
this implies $\mathbf x = 0,$ in particular, $z = 0,$
so Equation $(1)$ has no solution.
If $a',$ $b',$ and $c'$ are all non-zero but
do not all have the same sign, at least one of $a',$ $b',$ and $c'$ is positive and at least one is negative.
In that case the surface described by Equation $(4)$ is an infinite double right elliptical cone with its axis aligned with one of the rotated axes ($x',$ $y',$ or $z'$).
Depending on the shape of the cone and the rotation of the axes by the matrix $R,$ the intersection of this cone with the plane $z=1$ can be an ellipse, a parabola, or a hyperbola.
Now let's consider the case where $\Delta = 0$.
For the sub-case where exactly one of $a',$ $b',$ and $c'$ is zero, suppose $c'=0.$ Then if $a'$ and $b'$ have the same sign the solution of Equation $(4)$ is $x'=y'=0,$ a single line, which implies the solution of Equation $(1)$ is a single point or the empty set.
If $a'$ and $b'$ have opposite signs then Equation $(4)$ is equivalent to
$$
\left(\sqrt{\lvert a'\rvert}\;x'
       + \sqrt{\lvert b'\rvert}\; y'\right)
\left(\sqrt{\lvert a'\rvert}\; x'
       - \sqrt{\lvert b'\rvert}\; y'\right) = 0,
$$
which describes a pair of intersecting planes, so the solution of Equation $(1)$ is either one or two lines.
We get the same kinds of results if we suppose $a'=0$ or $b'=0.$
For the sub-case where two of $a',$ $b',$ and $c'$ are zero,
suppose $b'=c'=0.$ Then Equation $(4)$ is just
$$ a'x'^2 = 0,$$
whose solution set is a pair of parallel planes, so the solution of Equation $(1)$ is either two lines or the empty set.
We get the same kinds of results if we suppose $b'$ or $c'$ is the non-zero coefficient.
The final sub-case is that $a'=b'=c'=0,$ that is,
$D = 0,$ which implies that $M=0,$ in which case the solution set of Equation $(4)$ is the entire space, and the solution set of Equation $(1)$ is the entire plane.
At this point we have seen that if $\Delta \neq 0$ we have either an ellipse, a parabola, or a hyperbola,
whereas if $\Delta = 0$ we have either a pair of lines, a single line, a single point, the empty set, or the entire plane.
So the original statement is not quite complete, because it considered only a pair of lines as a possible solution when $\Delta = 0$.
But you could say in that case that if the solution is not trivial (empty or the entire plane) and is more than one point, it is a pair of lines which may or may not be the same line.

To classify the section of the elliptical cone in the case where $\Delta \neq 0$, let's look at the upper left $2\times2$ submatrix of $M.$ Let
$$
A = \begin{pmatrix} a&h \\ h&b \end{pmatrix}
\quad \text{and} \quad
\mathbf b = \begin{pmatrix} g\\f \end{pmatrix}.
$$
Then $M$ can be written as a block matrix,
$$
 M = \begin{pmatrix} A & \mathbf b \\ \mathbf b^\top & c \end{pmatrix}
   = \left(\begin{array}{cc|c}
       a&h&g \\ h&b&f \\ \hline g&f&c
     \end{array}\right).
$$
Note that $A$ is symmetric and so can be diagonalized by some orthogonal $2\times2$ matrix $R_2$.
If we let
$$
S = \begin{pmatrix} R_2 & \begin{matrix} 0\\0 \end{matrix} \\
              \begin{matrix} 0&0 \end{matrix} & 1 \end{pmatrix}
$$
then $S$ is a rotation around the $z$ axis and the upper left
$2\times 2$ submatrix of $S^\top MS$ is diagonal.
If $h^2 \neq ab,$ that is, if the determinant of $A$ is not zero, then the upper $2\times 2$ submatrix of $S^\top MS$ has non-zero entries on both diagonal elements and there is a translation parallel to the $x,y$ plane that zeros out the remaining off-diagonal elements of $S^\top MS$.
(See this answer to another question for an idea about how this can be done.)
In that case, setting $z = 1$, we get an equation of the form
$$ a'' x''^2 + b'' y''^2 + c'' = 0 \tag5$$
(in the translated coordinates $x'', y''$)
where
$$ \begin{vmatrix} a'' & 0 \\ 0 & b'' \end{vmatrix}
 = \begin{vmatrix} a & h \\ h & b \end{vmatrix}. $$
If the solution of Equation $(5)$ is not empty,
it is either an ellipse (when $a''$ and $b''$ have the same sign, which occurs if and only if the determinant of $A$ is positive, which occurs if and only if $h^2 < ab$)
or a hyperbola
(when $a''$ and $b''$ have opposite signs, if and only if the determinant of $A$ is positive, if and only if $h^2 > ab$).
If $h^2 = ab,$ then the determinant of $A$ is zero, and when we diagonalize it we get a matrix with only one non-zero element.
It is not then possible in general to zero out all off-diagonal elements of $S^\top MS$ by translation of coordinates parallel to the $x,y$ plane, but it is possible to zero out one element in the rightmost column and the corresponding element in the bottom row. Assuming the upper left element of $S^\top MS$ is non-zero (which we can accomplish by appropriate selection of the rotation $S$), by translation we can replace $S^\top MS$ with a matrix of the form
$$M''=
\begin{pmatrix} a''&0&0 \\ 0&0&f''\\ 0&f''&c''\end{pmatrix},$$
and the solution of $\mathbf x^\top M\mathbf x = 0$
has the same shape as the solution of
$$ a''x''^2 + 2f''y'' + c'' = 0. \tag6 $$
The condition $\Delta \neq 0$ implies that the matrix $M''$ has a non-zero determinant, which implies that $f'' \neq 0,$
and therefore the solution of Equation $(6)$ is a parabola.
In summary, in the case $h^2 < ab$ we have a transformed equation whose solution is an ellipse, in the case $h^2 > ab$ the transformed solution is a hyperbola, and in the case $h^2 = ab$ the transformed solution is a parabola.
The solution of the original Equation $(1)$ is the same shape as the transformed solution, but may be rotated and/or translated in the $x,y$ plane.
A: Let's go the other way:
Any pair of lines has equation $k(a_1 x+b_1 y+c_1)(a_2 x+b_2 y+c_2)=0,$ which corresponds to the matrix being $\begin{pmatrix}a_1 \\b_1\\c_1\end{pmatrix}\begin{pmatrix}a_2&b_2&c_2\end{pmatrix},$ which is of rank $1$ and hence has zero determinant. Also all the 2 by 2 minors are zero in particular $h^2-ab=0.$
Now any parabola can be written as a multiple of  $\frac{(a_d x+b_d y+c_d)^2}{a_d^2+b_d^2}=(x-f_x)^2+(y-f_y)^2$ or $-k\frac{(b_d x-a_d y)^2+((-2b_d^2f_x)-2a_d^2f_x-2a_dc_d)x+((-2b_d^2f_y)-2a_d^2f_y-2b_dc_d)y+(b_d^2f_y^2+a_d^2f_y^2+b_d^2f_x^2+a_d^2f_x^2-c_d^2)}{a_d^2+b_d^2}=0.$ That is the first minor $\frac{k^2}{(a_d^2+b_d^2)^2}\begin{pmatrix}b_d^2&-a_db_d\\-a_db_d&a_d^2\end{pmatrix}$ vanishes or $h^2=ab.$
More generally any ellipse can be given as having equation proportional to $e^2\frac{(a_d x+b_d y+c_d)^2}{a_d^2+b_d^2}=(x-f_x)^2+(y-f_y)^2$ with the squared eccentricity $0<e^2<1$ and any hyperbola by the same equation with $e^2>1$. Making the first minor
$\frac{k^2}{(a_d^2+b_d^2)^2}\begin{pmatrix}a_d^2(e^2-1)-b_d^2&a_db_de^2\\a_db_de^2&b_d^2(e^2-1)-a_d^2\end{pmatrix},$ which has determinant proportional to $(e^2-1)(a_d^2+b_d^2)^2.$
