Identifying the Plane at Infinity in the World Necessitates Determining the Affine Geometry of the World?

Question

Page 18 of my computer vision textbook, Multiple View Geometry in Computer Vision (Second Edition), by Hartley and Zisserman, states the following:

1.8 Auto Calibration

$\vdots$

Generally given three cameras known to have the same calibration, it is possible to determine the absolute conic, and hence the calibration of the cameras. However, although various methods have been proposed for this, it remains quite a difficult problem.

Knowing the plane at infinity. One method of auto-calibration is to proceed in steps by first determining the plane on which it lies. This is equivalent to identifying the plane at infinity in the world, and hence to determining the affine geometry of the world. ...

The textbook is freely available here.

It seems that the above excerpt is implying that identifying the plane at infinity in the world necessitates determining the affine geometry of the world. My question is, why is this the case?

I would greatly appreciate it if people could please take the time to clarify this. I do not have a strong background in geometry, so I would also appreciate it if people could please explain it carefully.

---

EDIT:

I will attempt to add some clarification.

What does "points at infinity" and "projective geometry" mean? How does projective geometry relate to Euclidean geometry?

Euclidean geometry describes angles and shapes of objects. Euclidean geometry is troublesome in one major way -- we need to keep making an exception to reason about some of the basic concepts of the geometry - such as the intersection of lines. In the context of $2$-dimensional geometry, two lines almost always meet in a point; we call those that do not "parallel". A linguistic device used in mathematics to get around this is to say that parallel lines meet "at infinity". However, this is not altogether convincing, and conflicts with another dictum - that infinity does not exist, and is only a convenient fiction. We get around this by enhancing the Euclidean plane by the addition of these points at infinity where parallel lines meet, and resolving the difficulty with infinity by calling them "ideal points".

By adding the points at infinity, the familiar Euclidean space is transformed into a new type of geometric object: projective space. Projective space is just an extension of Euclidean space in which two lines always meet in a point, though sometimes at mysterious points at infinity.

What does the "plane at infinity" mean?

This is clarified on page 2 of the text.

$(kx, ky, k)$ represents the same point for any non-zero value $k$. These points are represented by equivalence classes of coordinate triples, where two triples are equivalent when they differ by a common multiple. These are called the homogeneous coordinates of the point. Given a coordinate triple $(kx, ky, k)$, we can get the original coordinates back by dividing by $k$ to get $(x, y)$.

Although $(x, y, 1)$ represents the same point as the coordinate pair $(x, y)$, there is no point that correspond to the triple $(x, y, 0)$. If we try to divide by the last coordinate, we get the point $(x/0, y/0)$, which is infinite. This is how the points at infinity arise; they are the points represented by homogeneous coordinates in which the last coordinate is zero.

Once we have seen how to do this for $2$-dimensional Euclidean space, extending it to a projective space by representing points as homogeneous vectors, it is clear that we can do the same thing in any dimension. The Euclidean space $\mathbb{R}^n$ can be extended to a projective space $\mathbb{P}^n$ by representing points as homogeneous vectors. It turns out that the points at infinity in the two-dimensional projective space form a line, usually called the line at infinity. In three-dimensions, they form the plane at infinity.

What does "world" mean in this context?

In computer vision problems, projective space is used as a convenient way of representing the real $3$D world, by extending it to the $3$-dimensional projective space. Similarly, images, usually formed by projecting the world onto a $2$-dimensional representation, are for convenience extended to be thought of as lying in the $2$-dimensional projective space. In reality, the real world, and images of it, do not contain points at infinity, and we need to keep our finger on which are the fictitious points, namely the line at infinity in the image and the plane at infinity in the world.

How is "affine" used in this context?

In classical Euclidean geometry, all points are the same; there is no distinguished point. In other words, the whole of the space is homogeneous. When coordinates are added, one point is seemingly picked out as the origin. However, it is important to realize that this is just an accident of the particular coordinate frame chosen. We could just as well find a different way of coordinatizing the plane in which a different point is considered to be the origin. In fact, we can consider a change of coordinates for the Euclidean space in which the axes are shifted and rotated to a different position. We may think of this in another way as the space itself translating and rotating to a different position. The resulting operation is known as a Euclidean transform.

A more general type of transformation is that of applying a linear transformation to $\mathbb{R}^n$, followed by a Euclidean transformation moving the origin of the space. We may think of this as the space moving, rotating, and finally stretching linearly, possibly by different ratios in different directions. The resulting transformation is known as an affine transformation.

The result of either a Euclidean or affine transformation is that points at infinity remain at infinity. Such points are in some way preserved, at least as a set, by such transformations. They are in some way distinguished, or special in the context of Euclidean or affine geometry.

What is meant by "affine geometry" and "affine transformation"?

Take the point of view that the projective space is initially homogeneous, with no particular coordinate frame being preferred. In such a space, there is no concept of parallelism of lines, since parallel lines (or planes in the three-dimensional case) are ones that meet at infinity. However, in projective space, there is no concept of which points are at infinity - all points are created equal. We say that parallelism is not a concept of projective geometry. It is simply meaningless to talk about it.

In order for parallelism to make sense, we need to pick out some particular line, and decide that this is the line at infinity. This results in a situation where although all points are created equal, some are more equal than others. Thus, start with a blank sheet of paper, and imagine that it extends to infinity and forms a projective space $\mathbb{P}^2$. What we see is just a small part of the space, that looks a lot like a piece of the ordinary Euclidean plane. Now, let us draw a straight line on the paper, and declare that this is the line at infinity. Next, we draw two other lines that intersect at this distinguished line. Since they meet at the “line at infinity” we define them as being parallel. The situation is similar to what one sees by looking at an infinite plane. Think of a photograph taken in a very flat region of the earth. The points at infinity in the plane show up in the image as the horizon line. Lines, such as railway tracks show up in the image as lines meeting at the horizon. Points in the image lying above the horizon (the image of the sky) apparently do not correspond to points on the world plane. However, if we think of extending the corresponding ray backwards behind the camera, it will meet the plane at a point behind the camera. Thus there is a one-to-one relationship between points in the image and points in the world plane. The points at infinity in the world plane correspond to a real horizon line in the image, and parallel lines in the world correspond to lines meeting at the horizon. From our point of view, the world plane and its image are just alternative ways of viewing the geometry of a projective plane, plus a distinguished line. The geometry of the projective plane and a distinguished line is known as affine geometry and any projective transformation that maps the distinguished line in one space to the distinguished line of the other space is known as an affine transformation.

Answer 1

identifying the plane at infinity in the world, and hence to determining the affine geometry of the world.

Given a projective space ($\Bbb P^3$), we need to introduce a (3D) affine world in it. For this purpose we proceed as you wrote:

Take the point of view that the projective space is initially homogeneous, with no particular coordinate frame being preferred. In such a space, there is no concept of parallelism of lines, since parallel lines (or planes in the three-dimensional case) are ones that meet at infinity. However, in projective space, there is no concept of which points are at infinity - all points are created equal. We say that parallelism is not a concept of projective geometry. It is simply meaningless to talk about it.

In order for parallelism to make sense, we need to pick out some particular line, and decide that this is the line at infinity. This results in a situation where although all points are created equal, some are more equal than others.

So to introduce an affine geometry, we pick a (projective) plane in the projective space and cut the space along the plane, thus removing the points of the plane from the space and making them ideal points at infinity of the created world.

Start with a blank sheet of paper, and imagine that it extends to infinity and forms a projective space $\Bbb P^2$. What we see is just a small part of the space, that looks a lot like a piece of the ordinary Euclidean plane. Now, let us draw a straight line on the paper, and declare that this is the line at infinity.

I recall that projective lines and planes topologically are very different from their Euclidean counterparts. A projective line is topologically equivalent to a circle, and the projective plane even cannot be embedded in Euclidean 3D space without self-intersections. Topologically, it is convenient to see the projective plane as the closed disk $D$ with any two antipodal points identified. Its ground Euclidean plane is homeomorphic to the interior of $D$, and the line at infinity corresponds to the boundary of $D$ (with opposite points identified).

Answer 2

There are many definitions of a (real) affine space; for the sake of the answer, I will use the definition given in this Wikipedia article: A real affine space $A^n$ is a real $n$-dimensional vector space equipped with a simply-transitive action of the additive group ${\mathbb R}^n$ satisfying certain axioms: Every vector $v\in R^n$ defines a parallel translation $T_v$ on $A^n$.

Next, the standard definition of the real projective space $RP^n$ is the quotient of ${\mathbb R}^{n+1}-\{0\}$ by the equivalence relation: $v\sim w$ if there exists a scalar $r$ such that $av=w$ (i.e. $v$ and $w$ are linearly dependent). The projection $p: {\mathbb R}^{n+1}-\{0\}\to RP^n$ is called the projectivization.

What is a "hyperplane at infinity" in $RP^n$? It is any $n-1$-dimensional projective subspace $RP^{n-1}\subset RP^n$. In the end, it does not matter which one you take (the group of projective transformations acts transitively on the set of such subspaces). I will pick $RP^{n-1}$ which is the projection of the hyperplane $\{x_{n+1}=0\}$ (minus the origin) to $RP^n$ under the map $p$.

Now, consider the affine subspace $A=\{x_{n+1}=1\}$ in $R^{n+1}$. One verifies that the restriction of $p$ to $A$ is a homeomorphism, its image, is $$ RP^n \setminus RP^{n-1}. $$
Let us equip $A$ (hence, $p(A)$) with the structure of an affine space: The identification of $A$ with the vector space $R^n$ is given by $$ (a_1,...,a_n,1)\mapsto (a_1,...,a_n). $$ The action of the group $R^n$ on $A$ is given by: If $v=(v_1,...,v_n)\in R^n$ and $a=(a_1,...,a_n,1)\in A$ then $$ T_v: a\mapsto (a_1+v_1,...,a_n+v_n, 1). $$ Then one verifies that this action satisfies all the axioms listed in the Wikipedia article (this is a straightforward computation).

One can see that each $T_v$ acts as a projective transformation on $RP^n$ fixing $RP^{n-1}$ pointwise. Namely, elements of $PGL(n,R)$ are the projections of the matrices $g\in GL(n+1,R)$, acting on $RP^n$ by the rule: $$ g([v])=[gv], $$ where $[v]=p(v)$. The translation $T_v$ corresponds to the $(n+1)\times (n+1)$ matrix $$ a_{ii}=1, i=1,...,n+1,$$ $$ a_{in}=v_i, i=1,...,n $$ $$ a_{ij}=0, \hbox{otherwise}. $$

Just for completeness, let us identify some standard objects in $A^n=p(A)$:

1. Points in $A^n$ are $p(a)$, where $a\in A$. Equivalently, these are the elements of $RP^n$ which do not belong to $RP^{n-1}$ (the hyperplane at infinity).

2. Affine lines in $A^n$ are the projections $p(P)$ of 2-d vector subspaces $P\subset R^{n+1}$ which are not contained in the hyperplane $\{x_{n+1}=0\}$. In other words, these are nonempty intersections with $p(A)$ of projective lines in $RP^n$ (as they should be).

Hope it helps.

Edit. Your question in the post is:

It seems that the above excerpt is implying that identifying the plane at infinity in the world necessitates determining the affine geometry of the world. My question is, why is this the case?

When you say "necessitates" I read it "implies" since, clearly identifying a hyperplane at infinite does not "force" you to do anything. In my answer I explained exactly this:

The word "world" is poor man's substitute for "$RP^n \setminus RP^{n-1}$", where $RP^{n-1}$ is the hyperplane at infinity. In my answer I explained how one defines an affine structure on
$RP^n \setminus RP^{n-1}$ so that affine lines are parts of projective lines, etc.

I used coordinate to do so, which makes the discussion easy and algebraic. I understand why reading might tough, I was writing my answer assuming that you know some basic topology (say, introductory graduate course or even an advanced undergraduate course, depending on the univirsity). Otherwise, I would be forced to explain meaning of words such as "an equivalence relation", a "quotient space", etc.

In principle, what I wrote all can be done by synthetic projective geometry (using an axiomatic description of the projective space instead of working with the quotient on $R^{n+1}$). But this takes much more work. If you like the synthetic approach, read chapter 1 of

R. Hartshorne, "Foundations of Projective Geometry".