Find the locus of the vertices of the right circular cones that pass through the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1, z=0$ 
Prove that the locus of the vertices of the right circular cones that pass through the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1, z=0$ is $\frac{x^2}{a^2-b^2}-\frac{z^2}{b^2}=1, y=0$ or $\frac{y^2}{a^2-b^2}+\frac{z^2}{a^2}=-1, z=0$.  

EDIT:
Here $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1, z=0$ is the base of the cone. Let the vertex of the cone be $(x_1,y_1,z_1)$. Let the generator be $$\frac{x-x_1}{l}=\frac{y-y_1}{m}=\frac{z-z_1}{n}.$$
Then $z=0$ implies any point on ellipse be $(x_1-lz_1/n, y_1-mz_1/n,0)$. It lies on the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$ then the point satisfies the ellipse we get $$\frac{(nx_1-lz_1)^2}{a^2}+\frac{(ny_1-mz_1)^2}{a^2}=n^2$$ 
Eliminating $l,m,n$ we get the equation of the cone as $$\frac{1}{a^2}(zx_1-xz_1)^2+\frac{1}{b^2}(zy_1-yz_1)^2=(z-z_1)^2.$$ How to get the locus of vertex $(x_1,y_1,z_1)$?
Edit 2
How to get the locus of the vertices in the given two forms using purely mathematical way? 
 I not able to solve the problem. Please help. 
 A: In this answer, I answer the question "What is the cone of the conic section?" by considering Dandelin Spheres, which are tangent to the cone and to the plane of a conic section; the key fact is that Dandelin Spheres are tangent to the plane of a conic section at the foci of that conic.
The figure below, taken from that answer, shows the view of the plane perpendicular to an ellipse through its major axis. The ellipse itself appears only as $\overline{PQ}$ (its major axis) and the Dandelin Spheres have become "Dandelin Circles" $\bigcirc{R}$ and $\bigcirc{R^\prime}$. Point $C$, the apex of the cone, is the intersection of lines through $P$ and $Q$ that are tangent to these circles.

With that introduction, identifying the locus of $C$ is straightforward. We simply observe that
$$|\overline{CS}| = |\overline{CT}| \quad\to\quad |\overline{CP}|+|\overline{PF}| = |\overline{CQ}| + |\overline{QF}| \tag{$\star$}$$
Writing $m$ and $c$ for the ellipse's major and "focal" radii, $(\star)$ says
$$|\overline{CP}| - |\overline{CQ}| = |\overline{QF}| - |\overline{PF}| = (m+c)-(m-c) = 2c$$
That is, the difference of $C$'s distances from $P$ and $Q$ is a constant, so that $C$ lies on a hyperbola with foci $P$ and $Q$ that passes through $F$ and $F^\prime$. This hyperbola has focal radius $m$, transverse radius $c$, and conjugate radius $n := \sqrt{m^2-c^2}$ (equal to the minor radius of the ellipse). Its equation is
$$\frac{w^2}{c^2} - \frac{z^2}{n^2} = 1 \qquad\to\qquad \frac{w^2}{m^2-n^2} - \frac{z^2}{n^2} = 1 \tag{$\star\star$}$$
where the $w$-axis is horizontal in the figure. That axis is either the $x$-axis or the $y$-axis, according to which of $a$ or $b$ is bigger in the given ellipse equation,
$$\frac{x^2}{a^2} + \frac{y^2}{b^2} = 1$$
When $a$ is bigger, $\overline{PQ}$ aligns with the $x$-axis, so the figure shows the $xz$-plane; the locus of $C$ is
$$\frac{x^2}{a^2-b^2} - \frac{z^2}{b^2} = 1, \qquad y=0$$
When $b$ is bigger, $\overline{PQ}$ aligns with the $y$-axis, so the figure shows the $yz$-plane; the locus of $C$ is
$$\frac{y^2}{b^2-a^2} - \frac{z^2}{a^2} = 1, \qquad x=0$$
or, equivalently, with appropriate sign changes,
$$\frac{y^2}{a^2-b^2} + \frac{z^2}{a^2} = -1, \qquad x=0$$

Note. For the configuration shown in the figure, $C$ is on one semi-branch of the hyperbola, the "lower-right". Adjusting $\bigcirc{R}$ and $\bigcirc{R^\prime}$ to appear on the other sides of $\overline{PQ}$, and switching which circle is bigger, allows $C$ to travel throughout the entire hyperbola.)
A: You have a good start of getting an enveloping elliptic cone:
$$\frac{(Z x-X z)^2}{a^2}+\frac{(Y z-Z y)^2}{b^2}=(z-Z)^2$$
Rearrange to

$$
\begin{pmatrix}
  x & y & z
\end{pmatrix}
\begin{pmatrix}
  \frac{Z^2}{a^2} & 0 & -\frac{XZ}{a^2} \\
  0 & \frac{Z^2}{b^2} & -\frac{YZ}{b^2} \\
  -\frac{ZX}{a^2} & -\frac{ZY}{b^2} & \frac{X^2}{a^2}+\frac{Y^2}{b^2}-1
\end{pmatrix}
\begin{pmatrix}
  x \\ y \\ z
\end{pmatrix}+
2Zz-Z^2=0$$

Due to circular symmetry of right circular cone, the quadratic terms can be diagonalized into the form of $$\lambda x'^2+\lambda y'^2+\mu z'^2$$
which means the matrix has two equal eigenvalues.
That is
$$\det
\begin{pmatrix}
  \frac{Z^2}{a^2}-\lambda & 0 & -\frac{XZ}{a^2} \\
  0 & \frac{Z^2}{b^2}-\lambda & -\frac{YZ}{b^2} \\
  -\frac{ZX}{a^2} & -\frac{ZY}{b^2} & \frac{X^2}{a^2}+\frac{Y^2}{b^2}-1-\lambda
\end{pmatrix}=0$$
has a double root.

Also the vertex should lie on the mirror plane of the ellipse (i.e. either $X=0$, $Y=0$ or $Z=0$; that's only the case for right circular cone).

Case I: $X=0$
$$
\left( \frac{Z^2}{a^2}-\lambda \right)
\left[
  \lambda^2+
  \left( 1-\frac{Y^2+Z^2}{b^2} \right)\lambda-
  \frac{Z^2}{b^2}
\right]=0$$
We have two positive roots and one negative root, both factors share the same root $\lambda=\dfrac{Z^2}{a^2}$.
Now
$$\frac{Z^4}{a^4}+
  \left( 1-\frac{Y^2+Z^2}{b^2} \right) \frac{Z^2}{a^2}-
  \frac{Z^2}{b^2}=0 $$

$$\frac{Y^2}{a^2-b^2}+\frac{Z^2}{a^2} = -1$$
which is an imaginary ellipse provided $a^2 > b^2$.

Case II: $Y=0$
$$
\left( \frac{Z^2}{b^2}-\lambda \right)
\left[
  \lambda^2+
  \left( 1-\frac{X^2+Z^2}{a^2} \right)\lambda-
  \frac{Z^2}{a^2}
\right]=0$$
We have two positive roots and one negative root, both factors share the same root $\lambda=\dfrac{Z^2}{b^2}$.
Now
$$\frac{Z^4}{b^4}+
  \left( 1-\frac{X^2+Z^2}{a^2} \right) \frac{Z^2}{b^2}-
  \frac{Z^2}{a^2}
  =0 $$

$$\frac{X^2}{a^2-b^2}-\frac{Z^2}{b^2} = 1$$
  which is a hyperbola provided $a^2 > b^2$.

Case III: $Z=0$
$$\lambda^2
  \left( \frac{X^2}{a^2}+\frac{Y^2}{b^2}-1-\lambda \right) = 0$$
The double root is $\lambda=0$, the cone degenerates to a flat plane $z=0$.

The vertex is any point on
$$Z=0$$

Further points to be noticed:
The conics $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1 \cap z=0$ and $\frac{x^2}{a^2-b^2}-\frac{z^2}{b^2}=1  \cap y=0$ are known as the focal ellipse and focal hyperbola of confocal quadrics
$$\frac{x^2}{a^2+t}+\frac{y^2}{b^2+t}+\frac{z^2}{t}=1$$
where $a^2>b^2$.
See also another answer with picture of mine
.
A: Considering that


*

*the ellipse on the $x,y$ plane is symmetrical with repect to both axis, and also it is symmetrical with respect to the $z,x$ and $z,y$ planes.

*a circular cone whose axis is incident to the $x,y$ plane, will intercept on it a quadric that is symmetrical to the plane containing the cone axis and parallel to the $z$ axis. 


we conclude that the axis of the cone shall lie either on the $z,x$ or in the $z,y$ plane, and so shall thus be for the vertex .  
case: cone axis in the $z,x$ plane 
Let the vertex be $V=(x_v,0,z_v)$, and the unit vector defining the axis $\mathbb {a} =(cos \alpha, 0, sin\alpha)$.
A point $P$ on the circular cone will obey to
$$
\frac{{\mathop {VP}\limits^ \to  }}
{{\left| {\mathop {VP}\limits^ \to  } \right|}} \cdot \mathbf{a} = const. = \cos \beta 
$$
and in particular a point on the $x,y$ plane will obey to:
$$
\left( {\left( {x - x_{\,v} } \right)\cos \alpha  + \left( {0 - z_{\,v} } \right)\sin \alpha } \right)^{\,2}  = \left( {\left( {x - x_{\,v} } \right)^{\,2}  + \left( {0 - z_{\,v} } \right)^{\,2}  + y^{\,2} } \right)\cos ^{\,2} \beta 
$$
that is (duly excluding the cases $\beta=\pi /2$ and $\beta=\alpha$ which can be dealt apart)
$$
\begin{gathered}
  \left( {z_{\,v} ^{\,2}  + y^{\,2} } \right)\cos ^{\,2} \beta  =  \hfill \\
   = \left( {\cos ^{\,2} \alpha  - \cos ^{\,2} \beta } \right)\left( {x - x_{\,v}  + z_{\,v} \frac{{\sin \alpha }}
{{\left( {\cos \alpha  - \cos \beta } \right)}}} \right)\left( {x - x_{\,v}  + z_{\,v} \frac{{\sin \alpha }}
{{\left( {\cos \alpha  + \cos \beta } \right)}}} \right) \hfill \\ 
\end{gathered} 
$$
$$
\left( {1 - \frac{{\cos ^{\,2} \alpha }}
{{\cos ^{\,2} \beta }}} \right)\left( {x - x_{\,v}  - z_{\,v} \frac{{\sin \alpha }}
{{\left( {\cos \beta  - \cos \alpha } \right)}}} \right)\left( {x - x_{\,v}  + z_{\,v} \frac{{\sin \alpha }}
{{\left( {\cos \beta  + \cos \alpha } \right)}}} \right) + y^{\,2}  =  - z_{\,v} ^{\,2}  \tag{1}
$$
First step for this to represent the targeted ellipse is to get rid of the term in $x^1$, which means
that is shall be:
$$
 - x_{\,v}  - z_{\,v} \frac{{\sin \alpha }}
{{\left( {\cos \beta  - \cos \alpha } \right)}} =  - \left( { - x_{\,v}  + z_{\,v} \frac{{\sin \alpha }}
{{\left( {\cos \beta  + \cos \alpha } \right)}}} \right)
$$
i.e.
$$
 - z_{\,v} \left( {\frac{{\sin \alpha \;\cos \alpha }}
{{\cos ^{\,2} \beta  - \cos ^{\,2} \alpha }}} \right) = x_{\,v}  \tag{2}
$$
So that eq. (1) can be rewritten as:
$$
\left( {1 - \frac{{\cos ^{\,2} \alpha }}
{{\cos ^{\,2} \beta }}} \right)\left( {x + x_{\,v} \frac{{\cos \beta }}
{{\cos \alpha }}} \right)\left( {x - x_{\,v} \frac{{\cos \beta }}
{{\cos \alpha }}} \right) + y^{\,2}  =  - z_{\,v} ^{\,2} 
$$
leading to:
$$
\begin{gathered}
  \left( {1 - \frac{{\cos ^{\,2} \alpha }}
{{\cos ^{\,2} \beta }}} \right)x^{\,2}  + y^{\,2}  = x_{\,v} ^{\,2} \frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)}}
{{\cos ^{\,2} \alpha }} - z_{\,v} ^{\,2}  =  \hfill \\
   = x_{\,v} ^{\,2} \left( {\frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)}}
{{\cos ^{\,2} \alpha }} - \frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)^{\,2} }}
{{\sin ^{\,2} \alpha \;\cos ^{\,2} \alpha }}} \right) =  \hfill \\
   = x_{\,v} ^{\,2} \frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)}}
{{\cos ^{\,2} \alpha }}\frac{{1 - \cos ^{\,2} \beta }}
{{1 - \cos ^{\,2} \alpha }} \hfill \\ 
\end{gathered}  \tag{3}
$$
Comparing this to the target we obtain
$$
\left\{ \begin{gathered}
  x_{\,v} ^{\,2} \frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)}}
{{\cos ^{\,2} \alpha }}\frac{{\sin ^{\,2} \beta }}
{{\sin ^{\,2} \alpha }} = b^{\,2}  \hfill \\
  \left( {\frac{{\cos ^{\,2} \beta  - \cos ^{\,2} \alpha }}
{{\cos ^{\,2} \beta }}} \right) = \frac{{b^{\,2} }}
{{a^{\,2} }} \hfill \\ 
\end{gathered}  \right.
$$
With some algebraic manipulation
$$
\left\{ \begin{gathered}
  x_{\,v} ^{\,2} \frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)}}
{{\cos ^{\,2} \alpha }}\frac{{\sin ^{\,2} \beta }}
{{\sin ^{\,2} \alpha }} = b^{\,2}  \hfill \\
  x_{\,v} ^{\,2} \frac{{\cos ^{\,2} \beta }}
{{\cos ^{\,2} \alpha }}\frac{{\sin ^{\,2} \beta }}
{{\sin ^{\,2} \alpha }} = a^{\,2}  \hfill \\
  x_{\,v} ^{\,2} \frac{{\sin ^{\,2} \beta }}
{{\sin ^{\,2} \alpha }} = a^{\,2}  - b^{\,2}  \hfill \\
  \frac{{\left( {\cos ^{\,2} \beta  - \cos ^{\,2} \alpha } \right)}}
{{\cos ^{\,2} \alpha }} = \frac{{b^{\,2} }}
{{\left( {a^{\,2}  - b^{\,2} } \right)}} \hfill \\
  \left( {\frac{{\cos ^{\,2} \beta  - \cos ^{\,2} \alpha }}
{{\cos ^{\,2} \beta }}} \right) = \frac{{b^{\,2} }}
{{a^{\,2} }} \hfill \\
  \frac{{\cos ^{\,2} \beta }}
{{\cos ^{\,2} \alpha }} = \frac{{a^{\,2} }}
{{\left( {a^{\,2}  - b^{\,2} } \right)}} \hfill \\ 
\end{gathered}  \right.
$$
we finally arrive to rewrite the eq. (2) as:
$$
z_{\,v} ^{\,2}  = \frac{{b^{\,2} }}
{{\left( {a^{\,2}  - b^{\,2} } \right)}}\left( {1 - \frac{{\sin ^{\,2} \beta }}
{{\sin ^{\,2} \alpha }}} \right)x_{\,v} ^{\,2}  = \frac{{b^{\,2} }}
{{\left( {a^{\,2}  - b^{\,2} } \right)}}x_{\,v} ^{\,2}  - b^{\,2} 
$$
$$
\frac{{x_{\,v} ^{\,2} }}
{{\left( {a^{\,2}  - b^{\,2} } \right)}} - \frac{{z_{\,v} ^{\,2} }}
{{b^{\,2} }} = 1
$$
case: cone axis in the $z,y$ plane
The overall scheme is the same, just we have to exchange $x$ with $y$ and $a$ with $b$, giving
$$
\frac{{y_{\,v} ^{\,2} }}
{{\left( {b^{\,2}  - a^{\,2} } \right)}} - \frac{{z_{\,v} ^{\,2} }}
{{a^{\,2} }} = 1
$$
or if you prefer:
$$
\frac{{y_{\,v} ^{\,2} }}
{{\left( {a^{\,2}  - b^{\,2} } \right)}} + \frac{{z_{\,v} ^{\,2} }}
{{a^{\,2} }} =  - 1
$$
