1
$\begingroup$

I am trying to get my head around more formal concepts in geometry for the purpose of understanding gradient descent and natural gradients in machine learning. But I feel I still do not understand curvature. Can you check my following informal description and then answer several questions?

The most general coordinate system is a curvilinear system whose basis vectors vary throughout a space $\mathcal{W}$, both in direction, magnitude and non-orthogonality. Such a space can be considered to possess intrinsic (as opposed to extrinsic) curvature which is the stretching or compression of the curvilinear grid. This coordinate system is considered non-Euclidean as lines are not necessarily parallel.

Euclidean spaces have `parallel' coordinate systems and therefore have no intrinsic curvature. Included in this group are Euclidean curvilinear systems such as planar polars, spherical and cylindrical polars (all whose bases are orthogonal but can vary orientations depending on spatial location). The most specialised system is the Cartesian system whose unit vectors are orthogonal and constant throughout the space.

This paper does an excellent job of describing intrinsic curvature, in the context of gradient descent on a loss surface $\mathcal{L}(\mathcal{\mathcal{W}})$ parametrised by the space $\mathcal{W}$. If $\mathbf{w}{\in}\mathcal{W}$ then the distance metric of space $\mathcal{\mathcal{W}}$ as a function of the Reimannian metric tensor $G(\mathbf{w})$ is

\begin{align*} d_{\mathbf{w}}(\mathbf{w}{+}\delta\mathbf{w})^{2}=\delta\mathbf{w}^{\top}G(\mathbf{w})\delta\mathbf{w}. \end{align*}

If $G(\mathbf{w}){=}\mathbf{I}$, then the distance metric reduces to the Euclidean norm i.e. there is no intrinsic curvature and there is no specific `importance' to any direction. If the Reimannian metric tensor is a diagonal of positive constants then the dimensions of the parameter space are stretched or contracted e.g. a bowl loss function can be stretched into a long valley (and vice versa).

Q1. i) If $G(\mathbf{w})$ is anything other than the identity, is the space technically non-Euclidean? In particular, do planar polar coordinates describe a Euclidean space? I thought that planar polar coordinates were Euclidean, however as stated in the paper (section 3. natural gradient adaptation) the metric tensor for planar polars is $\text{diag}(G(\mathbf{w})){=}[1\,\,\,\, r^{2}]$!?

ii) Furthermore, looking at the coordinate mesh of planar polars, it seems that you can create this geometry by contracting a cartesian grid in a symmetric way around the origin so that radial lines are no longer parallel... How can a polar mesh be Euclidean of its radial components are not parallel?

Now I am also having difficulty understanding the interplay between intrinsic and extrinsic curvature. I understand that a flat space can still have intrinsic curvature described by the the Reimannian metric tensor - a flat plane can be stretched and contracted by adjusting the magnitude of the unit vectors in different locations in $\mathcal{W}$. I also understand that if you embed a space in a higher dimensional space, one can describe its extrinsic curvature.

Q2. i) Do inhabitants of a space detect intrinsic curvature but are unaware of extrinsic curvature? By stretching a sheet in its unit vector directions, am I not stretching the unit vectors (I assume inhabitants use unit vectors as their base unit of measurement)?

ii) Do inhabitants detect the orientation of their unit vectors? I assume that they do notice these changes w.r.t to a base Euclidean coordinate system?

iii) Do inhabitants get to see the metric tensor or only what the weighted norm result is?

Q3. i) Creating a torus by folding up a 2D sheet embedded in a 3D space causes intrinsic curvature (compression on the inner rim and expansion on the outer rim). Does this mean that some intrinsic curvature is caused by higher dimensional manipulation?

ii) Can all extrinsic manipulations that stretch manifolds be reduced to a manifold with flat geometry with just intrinsic curvature?

Q4. i) Another question about a sphere, similar to Q1. If inhabitants follow lines of longitude from north to south poles, they will diverge away from each other, then half way down they will be walking parallel (as they cross the equator), finally coming together at the north pole. These lines are not parallel. How can this be Euclidean geometry?

$\endgroup$
1
$\begingroup$

Your formula $d_{\mathbf{w}}(\mathbf{w}{+}\delta\mathbf{w})^{2}=\delta\mathbf{w}^{\top}G(\mathbf{w})\delta\mathbf{w}$ is wrong (or maybe true on the tangent space rather than our manifold itself). The paper says "for small vectors $\delta \mathbf{w}$", by which they mean that it is true in the limit $\delta \mathbb{w} \to 0$. If you multiply $\delta \mathbf{w}$ by $h$ on both sides, divide both sides by $h^2$, and take the limit $h \to 0$, you get the correct formula. You can then use integration to compute the lengths of curves in your manifold, or define geodesics as curves which are locally the shortest. The distance between two points would be then the length of the shortest geodesic between them.

Let's assume two dimensional surfaces for simplicity, so that the intrinsic curvature is just one number (the Gaussian curvature), $K$.

Q1. The relationship between $K$ and $G$ is much more complex. $K$ can be computed from the (second) derivatives of $G$, see e.g. here: Calculating Gaussian curvature given first fundamental form only. It is true that $K=0$ is 0 if $G(\mathbf{w}){=}\mathbf{I}$ everywhere, but this is also the case whenever $G$ is constant. It is also true in some cases when $G$ is not constant, for example, for the polar coordinate system on the Euclidean plane.

If at some point $\mathbf{w}$ we have $G(\mathbf{w}){=}\mathbf{I}$, this says nothing about the curvature at $\mathbf{w}$ -- in fact, you could use another parametrization to make $G(\mathbf{w}) = \mathbf{I}$. On the other hand, the Gaussian curvature will not change if you change the parametrization in such a way that the lengths of curves are still computed correctly. (See theorema egregium.)

Q2. This is not really a mathematical question... assuming the physics of their world only depends on the distances between points, they cannot experience extrinsic curvature. They can measure lengths of their curves, they can create some coordinate system, find the $G$ according to their coordinate system, and compute the Gaussian curvature based on that. Their $G$ might not be the same as the one as you are using (for example, they might use a polar coordinate system while you use a Cartesian one). However, the areas and lengths of curves as computed according to their system will be the same as the ones you are using. Also, the Gaussian curvature at a given point will be the same.

Gaussian curvature measures effects such as:

  • do close "parallel" geodesics diverge (K<0) or converge (K>0) or stay parallel (K=0)?

  • is the length of a perimeter of a circle of small radius $r$ equal to $2\pi r$ (K=0), smaller ($K>0$) or greater ($K<0$)?

  • if you draw a triangle out of three geodesics, its sum of angles will be $\pi$ degrees plus the integral of the Gaussian curvature inside the triangle. For example, on a sphere of radius 1, the Gaussian curvature is 1 everywhere, and the triangle with vertices at the North pole and two points at the equator 90 degrees apart has the integral of curvature = area = $\pi/2$, and the sum of angles is $3/2 \pi$. (See the Gauss-Bonnet theorem.)

Q3. If you try to do this from a piece of paper, you will get some creases. However, you will not get any creases if you make a cylinder or a cone. This is because cylinder and cone have Gaussian curvature 0 everywhere.

Q4. The geometry of the sphere is non-Euclidean. You can embed a surface with intrinsic non-Euclidean geometry in an Euclidean space of a higher dimension, at least locally.

$\endgroup$
  • $\begingroup$ Thank you for answer. You have cleared up a few things. I'm still not understanding Q2. Let's resolve this by considering a flat Cartesian 2D plane. If I shrink and stretch this surface (in the 2D plane so that it still has 0 extrinsic curvature) described by Gaussian curvature, then the curvilinear unit vectors will go from being constant orthogonal to varying everywhere. Do flat people traverse their plane along its unit vectors and therefore unaware of intrinsic curvature? Is intrinsic curvature of this flat plane defined as a relative property to the original Cartesian plane? $\endgroup$ – rnoodle Oct 13 '18 at 8:13
  • $\begingroup$ Assuming their physics are based on the distances in the $\mathbb{R}^3$ world that their plane is a part of, the flat people would traverse the plane along the geodesics. Since they live on a weirdly parametrized plane, the geodesics would be straight lines (for an external observer, not if you draw graph of coords in the parametrization). If you look at the lines the planes take on a Mercator map, you will see that they appear curved towards the poles, but this is because that's how geodesics look on the Mercator map. The same paths drawn on a spherical model of the globe will look straight. $\endgroup$ – Zeno Rogue Oct 13 '18 at 10:09
  • $\begingroup$ (Geodesics on a sphere look like straight lines on a map based on the gnomonic projection, though.) Likewise, if you stretch your plane so that the original horizontal lines become circles around the center and original vertical lines become rays from the center (i.e., the original coordinates represent the polar coordinate system), the geodesics will look like straight lines for the external observer, but if you draw a graph of their $(r,\phi)$ coordinates in a rectangular coordinate system, they will not look straight. $\endgroup$ – Zeno Rogue Oct 13 '18 at 10:20
  • $\begingroup$ The Gaussian curvature curvature does not depend on the parametrization, just on distance measurements, so if the surface is the (weirdly parametrized) plane, it will still equal 0 everywhere. $\endgroup$ – Zeno Rogue Oct 13 '18 at 10:33
  • 1
    $\begingroup$ According to the formula (16) in the paper, every step is given as a vector in the parametrization, so this corresponds to a straight line in the parameter space, rather than a geodesic. It would be possible to make the procedure more intrinsic by making steps on a geodesic. I guess it works better in many cases -- for example, in this paper arxiv.org/pdf/1806.03417.pdf the authors do descent on geodesics, which is an improvement from an earlier version which used straight lines in the Poincaré model. In general, it could be much more complex computationally, so not necessarily good. $\endgroup$ – Zeno Rogue Oct 13 '18 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.