The 2-norm is convenient, but it is only special up to linear isomorphism. If one were so inclined on could use any norm of the form $|(x,y)|=a^2x^2+bxy+c^2y^2$ for non-zero $a,c$ and such that $b^2-4ac>0$. These are precisely the norms that can be obtained from the 2-norm via a linear isomorphism, and it is clear that instead of a unit circle they give us unit ellipses.
One way to think about these other norms is the following. When measuring distance with a ruler, the markings on the ruler remain the same no matter how you turn it about. This corresponds to the standard 2-norm. For an ellipse norm (or any norm), what happens is that the markings on the ruler shift as you rotate the ruler (so it's a ruler with an orientation sensor). Nevertheless, the nice norms are the ones for which if you fixed a pencil on one of the shifting marks, as you rotate the ruler around a central point, you would get an ellipse.
So your question is what distinguishes ellipses from all other shapes in the plane. The answer is that for any line through the center of an ellipse, there is a reflection across the line that sends the ellipse to itself. Note that there are many reflections across a line: for the x-axis for example, we can reflect as $(x,y)\mapsto(x,-y)$ or $(x,y)\mapsto(x-y,-y)$. One is a 90 degree reflection, the other is a 45 degree reflection across the x-axis. For the ellipse, the reflection across a line $\ell$ through the center of the ellipse is done along the line tangent to the intersection point(s) of $\ell$ and the ellipse. Since for a circle, the tangent line is always perpendicular to a radial line, the reflections are all perpendicular. In fact, it is always the case that that a reflection across a line $\ell$ that preserves a unit shape will be along the line tangent to the intersection point(s) of the line with the unit shape.
The special property of ellipses is that they are the only shapes with the property that there is a reflection preserving the ellipse across every radial line. This property has important implications for the following definition of angle measure: the measure of angle ABC is the arclength of the piece of the unit ellipse centered at B that is bounded between the rays BA and BC. This mirrors exactly the definition of angle with standard Euclidean distance. It may be useful to go through the list of the following results with the circle in mind as an example. Given the special property of ellipses, we can prove the following:
- Angles, defined as arc-lengths of the unit ellipse, are additive.
- A triangle with two equal sides has corresponding equal angles. The median to the base of such an isosceles triangle is the angle bisector and the orthogonal projection, where two lines are orthogonal if the angle between them is half the perimeter of the ellipse.
- Perpendicular bisectors exist, so any triangle ABC can be inscribed in an ellipse.
- If A,B,C are three points on unit ellipse with center O, then angle ABC is half of angle AOC.
- The angles in a triangle add up to the perimeter of an ellipse.
- Given an angle ABC we can define the rotation along that angle as the composition of any two (distance-preserving) reflections across lines intersecting at B and meeting at an angle half of that of ABC.
- Using rotations and reflections, we can prove the similarity theorems for triangles.
- We can prove the Pythagorean theorem as follows. Given right angle ABC, drop the perpendicular BD, use additivity of angles and the fact that the angles add up to the perimeter of an ellipse to show that angle ABD=ACB, angle CBD=CAB, and then use similarity of ADB to ABC to BDC to derive the Pythagorean theorem.
Once we have the Pythagorean theorem, it is easy to derive a linear isomorphism onto the Euclidean plane that sends the unit ellipse to the unit circle, and hence makes distances and angles as we are used to them.