# Transforming a 3d point to a 2d plane then back again to a 3d point

For 3d software, in the code, I'm changing a 3d point to a 2d point on a 2d plane, which represents a screen view, with the following method:

$$x, y, z$$ = given point in the 3d system.

$$(X_u, Y_u, Z_u)$$ = The transform for the 2d view plane Y-direction/upwards vector relative to the 3d system.

$$(X_r, Y_r, Z_r)$$ = The transform for the 2d view plane X-direction/rightwards vector relative to the 3d system.

View plane values: $$x_1, y_1, z_1$$

Equations:

$$x_1 = X_r x + Y_r y + Z_r z$$

$$y_1 = X_u x + Y_u y + Z_u z$$

$$z_1 = 0$$ Note that the software still tracks $$z_1$$ even though it always equals $$0$$

Now, I need to determine how to reverse this and determine the 3d point $$x, y, z$$. But I do not have the original 3d point. I do have values for:

The transforms for the 2d plane: $$(X_u, Y_u, Z_u)$$ and $$(X_r, Y_r, Z_r)$$

I, of course, have: $$x_1, y_1, z_1$$

I also happen to have the 3d x, y, z (say, $$x_2$$, $$y_2$$, $$z_2$$) values for a second point on the 2d plane (being the 3d values, this point has not been transformed, just like $$x, y, z$$ have not been transformed) The second point might be useful because $$z_1$$ was made to equal $$0$$. Though, please note, for my purposes, it should be okay if we pretend $$z_1 = 0$$ by way of a transform, so when it is reversed, the 3d point will actually lie on the 2d plane.

How can this be done?

• It can't be done with the information you gave. There is an entire line in your 3d space that gets projected onto the same point $(x_1,y_1,0).$ Unless there is some special relationship between the second 3d point and the first one that you've forgotten to mention, that point does not give you any clue about which of many possible points got projected to $(x_1,y_1,0).$ – David K Feb 25 at 19:11
• Look at it this way: what you're asking is like taking a single photograph of the night sky, pointing at a particular star, and asking how far away it is. I can give you the complete information about another star in the photograph (including distance, which gives you complete 3d coordinates for that star), even give you distances to a hundred other stars in the same photo, and you still won't know the distance to the first star. – David K Feb 25 at 19:15
• What if I had the value for $z_2$? Which, oddly, I should, because I have a second point that lies on the 2d plane but which is expressed in the 3d coordinate system. – Edward Bagby Feb 25 at 19:20
• I'm more concerned with transforming $x_1$, and $y_1$ back to their original x,y,z values. See my edited version, which indicates we can use $z_1 = 0$ when we transform it back. – Edward Bagby Feb 25 at 19:23
• None of this is any good. I'm sorry. This is a classic problem of reconstructing 3d data from a 2d image. If you had two different projections of the original point, or if you had some other information such as the distance between the two points in their original 3d coordinates, you might have something. But you need something that is uniquely about the original 3d point that cannot be true of any other point and is more than just the projected $(x_1,y_1,z_1)$ coordinates. – David K Feb 25 at 21:12

A pure 3D rotation matrix is orthonormal, $$\mathbf{R} = \left[ \begin{matrix} \hat{e}_1 & \hat{e}_2 & \hat{e}_3 \end{matrix} \right ] = \left [ \begin{matrix} x_1 & x_2 & x_3 \\ y_1 & y_2 & y_3 \\ z_1 & z_2 & z_3 \end{matrix} \right ]$$ where the three (column) vectors form the basis: $$\begin{array}{lll} \hat{e}_1\cdot\hat{e}_1 = 1, ~ & \hat{e}_1\times\hat{e}_1 = 0 \\ \hat{e}_1\cdot\hat{e}_2 = 0, ~ & \hat{e}_1\times\hat{e}_2 = \hat{e}_3 \\ \hat{e}_1\cdot\hat{e}_3 = 0, ~ & \hat{e}_1\times\hat{e}_3 = -\hat{e}_2 \\ \hat{e}_2\cdot\hat{e}_1 = 0, ~ & \hat{e}_2\times\hat{e}_1 = -\hat{e}_3 \\ \hat{e}_2\cdot\hat{e}_2 = 1, ~ & \hat{e}_2\times\hat{e}_2 = 0 \\ \hat{e}_2\cdot\hat{e}_3 = 0, ~ & \hat{e}_2\times\hat{e}_3 = \hat{e}_1 \\ \hat{e}_3\cdot\hat{e}_1 = 0, ~ & \hat{e}_3\times\hat{e}_1 = \hat{e}_2 \\ \hat{e}_3\cdot\hat{e}_2 = 0, ~ & \hat{e}_3\times\hat{e}_2 = -\hat{e}_1 \\ \hat{e}_3\cdot\hat{e}_3 = 1, ~ & \hat{e}_3\times\hat{e}_3 = 0 \\ \end{array}$$ The nice property of such matrices is that their inverse is their transpose: $$\mathbf{R}^{-1} = \mathbf{R}^{T} = \left [ \begin{matrix} x_1 & y_1 & z_1 \\ x_2 & y_2 & z_2 \\ x_3 & y_3 & z_3 \end{matrix} \right ]$$ which is also an orthonormal matrix itself. So, to reverse a pure rotation, you just apply the transpose of the matrix: $$\vec{a} = \mathbf{R}\vec{b} \quad \iff \quad \vec{b} = \mathbf{R}^{T} \vec{a}$$
The problem with recovering 3D position from a 2D image is that the third component (depth or distance or $$z$$) is unknown. This means that each 2D point can be only converted to a line, perpendicular to the original 2D plane, passing through the point on that 2D plane.