Odds of ellipse from five random points

It's known that five points determine a conic section. Five random points can go right into the $6\times6$ matrix, and then the $A x^2 + B xy + C y^2$ part can be looked at. If $B^2-4AC<0$, it's an ellipse. Five random points will almost never produce circles or parabolas, so the results will be ellipses and hyperbolas. What are the odds of an ellipse?

In a random run of 100000 trials, I got 27974 ellipses. "It's less than $e/10$," seems like a solid answer. Anyone have anything more specific?

EDIT: As Oscar points out, I should have said "It's more than $e/10$." In my trial, real-values points were randomly picked from a unit square. Square Triangle Picking methods might be applicable.

EDIT2: Aretino points out that odds of a convex pentagon are $49/144≈0.34$. So how can points making a convex pentagon give a non-ellipse? Here's a picture. With the red points fixed, the black points are outside of the convex hull yet still yield a non-ellipse. EDIT3: That spray of points above goes back to Newton, Philosophiae naturalis principia mathematica, 1687, where he solved the 4 point parabola (another version). If a point is between one of the two parabolas and the degenerate lines, then it gives a hyperbola.

• Uh ... $27974/100000>e/10$. $e=2.71828...$. – Oscar Lanzi Aug 16 '17 at 16:33
• Good eye, Oscar. I'm just being stupid. – Ed Pegg Aug 16 '17 at 16:35
• What do you mean by random points? – Philip Roe Aug 16 '17 at 16:43
• Is there any reason to believe that the domain from which you pick the points does not make a difference? Ellipseness is affine-invariant, so at least we know that points picked uniformly from any parallelogram will have the same probability, but I'm not confident about saying the same for points picked from, say, a triangle, or from a normal distribution. – Rahul Aug 16 '17 at 16:52
• I found at this page that the probability that five points chosen at random inside a square are the vertices of a convex pentagon is $49/144\approx0.34$. Not far from the results you got, but definitely not the same. – Aretino Aug 16 '17 at 18:28

Let $x$ be the probability of getting an ellipse. For the five points to form a convex polygon, they must either lie on an ellipse (with probability $x$) or lie on the same branch of a hyperbola: if each point has probability $1/2$ of lying on either branch of the hyperbola this should happen with probability ${1\over16}(1-x)$.
As the probability that five points chosen at random inside a square are the vertices of a convex pentagon is $49/144$, we have the equation $$x+{1\over16}(1-x)={49\over144}, \quad\hbox{whence:}\quad x={8\over27}\approx0.296.$$ This result is however a little higher than simulations suggest.
• I supposed that each point has probability $0.5$ of lying on either branch of the hyperbola. But that might not be exactly true. – Aretino Aug 16 '17 at 19:35
• I've just run a simulation taking a triangular domain: the probability turns out to be around $0.22$, while the same reasoning as above gives in this case ${7\over27}\approx0.26$. – Aretino Aug 16 '17 at 20:15
• I must correct the simulation result in my previous comment: the result for a triangle is about $0.24$. I was using by mistake a non-uniform distribution. – Aretino Aug 17 '17 at 6:03