# 3D projection coordinates onto 2D plane to determine transformation matrix?

I'm not sure if there is an actual solution to this problem or not, but thought I would give it a shot here to see if anyone has any ideas. So here goes:

I basically have three vertices of a rigid triangle with known 3D coordinates. The vertices are projected onto a 2D plane (by projection, I mean that each vertex would basically have a fixed line drawn from it to the 2D plane, and that "line" would also stay rigid to the triangle so that the lines would move along with the triangle if it is transformed), in which I also know the 2D coordinates. A transformation matrix is applied to the original three points (can be a combination of rotation and translation) and I now know the new 2D projection coordinates.

Is it possible to obtain either the unknown transformation matrix or the new coordinates? Any ideas are much appreciated. Thanks!

• wlog choose x-y plane as plane of projection. It is the$Triangle shadow$ problem where the orientation of of plane needs to be determined using cross product of any 2 edges. Sep 2, 2015 at 18:57

Here's a way of doing what I think you want to do.

First, find two perpendicular unit-vectors that are parallel to the plane (i.e. perpendicular to the normal vector). You might want to use Gram-Schmidt here. Call the vectors $u_1$ and $u_2$. These are the "axes" along the plane you're projecting onto.

Any point on the plane can be written in the form $a_1u_1 + a_2u_2$ for some coordinates $(a_1,a_2)$. To find the coordinates of the orthogonal projection of a vector $x$ onto your plane, calculate $$a_1 = u_1^Tx\\ a_2 = u_2^Tx$$ Or, all together: $$\pmatrix{a_1\\a_2} = \pmatrix{u_1^T\\u_2^T} \pmatrix{x_1\\x_2\\x_3}$$

• I don't think this is exactly what I am looking for, but I appreciate the attempt. I will try to explain the question a little better: Imagine a tetrahedron that has a plane going through it. I know the 2D coordinate positions of the edges intersecting the plane (thus there will be 3 edge intersections for a tetrahedron). I also know 3 of the 3D vertex positions. Now, the tetrahedron is rotated/translated with the plane remaining stationary. I now again know the 2D intersection locations on the plane, but would like to know the new 3D vertex locations, or the rotation/translation matrix. Aug 26, 2015 at 5:09
• Okay... perhaps you mean to say that you're contracting a triangle towards a point (your $4$th vertex of the tetrahedron) rather than projecting in the usual sense. Aug 26, 2015 at 10:48
• Here's what I think you mean: start with a triangle. Add a fourth vertex to make a tetrahedron. Draw a plane though that tetrahedron. The intersection of the edges of the tetrahedron with the plane are the vertices of the projected triangle on the plane. Is that right? Aug 26, 2015 at 10:50
• Yes, that sounds correct to me. So then I know the coordinates of the projected triangle (in reference to the 2D plane), as well as the three 3D coordinates of the original triangle. That original triangle is then rotated/translated in some unknown way, and I get back just the 2D projected triangle coordinates. I would then like to know the new 3D vertex locations, or the rotation/translation matrix. Is this making sense? Aug 26, 2015 at 18:17
• That sounds similar to what I was saying, I think we are on the same page. So yes, I would say that the triangle is being projected onto the z=0 plane. Aug 27, 2015 at 17:38

In general, the answer to your problem is no, it is not possible to do so. There are however variants that are possible if we know more about the transformation.

Both I get into detail, some notation and two observations:

Notation: We know 3 points, $a$,$b$,$c$ and a projection $P$ onto a 2d plane through the origin (so $P$ is a matrix with $P^2=P$ and $P$ is of rank 2). There is an unknown matrix $M$, but we know the three points $PMa, PMb$ an $PMc$.

1. I assume your original points $a$,$b$,$c$ are linearly independent, otherwise (for example if one of them is (0,0,0)) it will clearly not work.

2. It does not matter if we recover the 3d image points $Ma,Mb$ and $Mc$ or the transformation matrix. If we have the matrix we can apply it to $a,b,c$ in order to get the points. If we have the points, we can recover the matrix, since $a,b,c$ form a basis and any matrix is uniquely given by its image in any basis (see footnote 1).

Now to the actual problem. By a simple heuristic, we know three 2d coordinates, that is 6 independent values (the initial points $a,b,c$ do not count, see footnote 2), but we want to recover all 9 independent entries of $M$. There is obviously a mismatch here. This is no proof, but it tells us to search for a counterexample:

The problem essentialy is that we do not know the height the points $Ma,Mb$ and $Mc$ have above the plane. In fact let $v\neq 0$ be in the kernel of $P$, that is $Pv = 0$ (such a $v$ exists, since $P$ is of rank 2). Let $A$ be a matrix whose collumns are all multiples of $v$. Then the image of $A$ is the kernel of $P$ and we have $$P(M+A) x = PMx+PAx = PMx$$ for any $x$. So using only the data given we cannot distinguish $M$ and $M+A$ as transformations, thus the answer is no, it is not possible.

As I said in the beginning, however there variants, for which this is possible. For example, if we know that $M$ is a rotation, we search for fewer independent values, so it should be possible. As said in the footnotes, we know the matrix $PM$. Along the axis of rotation we know that $Mx=x$ and thus $PMx=Px$, which is a linear equation with a 1d space of solutions (except when $M$ is the identity, but that is exactly the case if $PM=P$, so we can exclude this) which then give us this axis.

Let us call the axis $v$. Now we have two cases:

1. The axis of rotation does not lie in the plane on which we project. Then $P$ restricted to vectors from the plane of rotation $v^\perp$ is invertible, so we can recover how $M$ transforms the plane $v^\perp$ and thus recover $M$.

2. The axis of rotation lies in the plane on which we project. Then there is a vector $x$ in $v^\perp$ such that $PMx=0$, since $x$ is rotated to a vector perpendicular to the plane. But then we can find $x$ and recover the angle of rotation this way, also finding $M$.

Footnotes:

1.More detail, by linear independence, there are $r,s,t$ such that $(1,0,0)^T=r a+sb+tc$, but then $M(1,0,0)^T = rM a+sMb+tMc$ so we recover the first collumn of $M$. Then continue in the same way for $(0,1,0)$ and $(0,0,1)$.

1. The points $a,b,c$ do not contain any information about $M$, they only serve to determine the images. However note that using the same idea as in footnote 1, we can actually recover the matrix $PM$ from the initial information. We cannot get $M$ from this, since $P$ is of rank 2 and thus not invertible.