Do 4 points in ${\mathbb R}^2$ in convex position define a unique elliplse that passes through those 4 points? So it takes 3 distinct points in the plane, that are not collinear, to define a unique circle that passes through the points.
So what about ellipses?
Arguing naively in terms of degrees of freedom doesn't seem to help too much, because for a circle we have 3 degrees of freedom (2-d center coordinates and 1-d radius), and yet it takes 3 2-d points (6 degrees of freedom), not 2 2-d points (4 degrees of freedom), to define a unique circle that passes through the points. I haven't studied conic sections much so I apologize if this question is trivial, but if we have 4 2-d points in convex position (i.e. each point is a vertex of the convex hull) does this define a unique ellipse that passes through the 4 2-d points? Or do we sometimes need 5 or more 2-d points in convex position?
 A: Not quite. Think of the vertices of a square; a tall and narrow ellipse passes through them, but so does a short and wide ellipse. So you don't get uniqueness. 
When you (naturally) ask about 5 points, it turns out that there is a unique conic containing any five points...but it's not necessarily an ellipse, even if the points form a nice convex set. Why?  Think of 5 points on a parabola. The unique conic that fits these is...that parabola! So there's no ellipse that passes through them. 
The projective space of conics is a fascinating introduction to algebraic geometry. The book on Projective Geometry by Pierre Samuel is a pretty nice intro if your other math skills are rock solid, but "other" includes abstract algebra in this case, so it'll be a while before you're ready to read it. 
Post-comment remarks
Let me just add another remark here, which hints at the projective geometry thing. Suppose that 
$$
H_1(x, y) = Ax^2 + 2Bxy + Cy^2 + Dx + Ey + F
$$
and that the four points $(x_i, y_i)$, $i = 1, 2, 3, 4$, have the property that 
$$
H_1(x_i, y_i) = 0 (i = 1, 2, 3, 4).
$$
Then the equation
$$
H_1(x, y) = 0
$$
defines a conic containing the four points. 
Now suppose that $H_2$ is another such polynomial (quadratic in $x$ and $y$) and that the equation $H_2(x, y) = 0$ is satisfied by the same four points. (Think of $H_1 = 0$ as defining the red ellipse in @muaddib's answer, and $H_2 = 0$ as defining the blue one.)
Then for any $t$,  the polynomial
$$
Q_t(x, y) = (1-t)  H_1(x, y) +  t H_2(x, y)
$$
also is zero at the four points. So in the "space of all conics", if two conics $C_1$ and $C_2$ pass through four points, so do all conics on the "line between $C_1$ and $C_2$" (i.e., those like $Q_t$ above).
To close the argument:
In general, if you have four points $P_1, P_2, P_3, P_4$, and they lie on an ellipse, $E$, you can pick a 5th point, say $R_0$, not close to any of the $P_i$, that's also on that ellipse.  Now you can move $R_0$ very slightly to get a new point $R_1$, and consider the conic $C$ passing through 
$$
P_1, P_2, P_3, P_4, R_1.
$$
If $R_1$ is close enough to $R_0$, then the conic $C$ is very close to the original ellipse, and must therefore also be an ellipse (a statement that needs proving, by the way -- it's not at all obvious!). 
But now you can take the quadratic for $E$, say $H_1$, and the quadratic for $C$, say $H_2$, and form a combination like $(1-t)H_1 + tH_2$, where $t$ is any real number, to get another conic passing through the four points, so there's a whole infinity of conics passing through these points. By the same argument as above -- the "not at all obvious" one -- infinitely many of these conics must also be ellipses.
In short:  no, even in the case where the four points are in "convex position" and there's an ellipse through them, there's always another ellipse through them, and indeed, infinitely many others.
A: This argument is taken from How many points does it take to define...
The picture says it all:


Suppose that we have K points, and we want these points to define only a single ellipse. If the points do define two ellipses, then the points must be found on the circumference of both – so they must be on the intersections between the two ellipses. Two ellipses intersect at most at four points – so if we have only four points, we are not able to resolve the ambiguity between the two. In order for there to be a contradiction, there must be at least five distinct points. Note that this proof doesn’t say that five points are enough; it only says that four are not.

A: This is an image to accompany a point I was making in a comment:

Four points ---here, the cyan ones at the corners of a square near the center--- don't just determine infinitely-many ellipses; they determine infinitely-many hyperbolas, too. (No parabolas this time.) Moreover, those ellipses and hyperbolas cover disjoint regions of the plane, in such a way that each point on the plane (other than the first four) lies on exactly one curve. This is what it means for a fifth point added to the collection to determine one of those curves uniquely. This is also why there has to be a "special" condition (namely: in which region to place the point) to guarantee that the fifth point determines the specific type of curve (ellipse or hyperbola) you might want. 
