# Applying 3D rotations to align two points in front plane

I've been trying to learn some 3D (euclidean) geometry techniques lately and one thing I like to do to wrap my head around things is to reduce 3D problems to 2D problems (wherever possible).

One particular instance goes like this: say I have 2 points in space, A and B. I want to apply a series of rotations to bring both A and B to the front plane (more technically, the z coordinate of both the points should become zero after the rotations). To me, it seems like this should always be possible because 3 points always form a plane (A, B and the origin).

I tried to write out what happens when applying a rotation along the X and Y axis to the two points (rotation along the Z axis seems unnecessary since it doesn't change the z coordinate). After multiplying the rotation matrices and making the resulting z coordinates zero, I get two equations in 4 unknowns (the cos and the sin of the rotation along X and along Y respectively) and then two more equations from the trigonometric identity between sin and cos.

Therefore, I get to 4 equations with 4 unknowns which sounds great at first, but the system is not of linear equations, but quadratic equations. After some searching, I found no easy way of solving systems of quadratic equations. Applying substitutions will lead to a lot of squaring in order to get rid of square roots, which will in turn lead to high degree polynomials, which are difficult to solve.

I'm curious to know if there's a different approach to this problem, or if there's something which would make my approach more feasible (the problem seems like it should have a beautiful solution).

• Form the rotation matrix that takes the cross product of $A$ and $B$ to $(0,0, |A\times B|)$. – John Douma Apr 5 '18 at 23:41

It shouldn’t be too surprising that you’re getting an underdetermined system of equations since there’s an infinite number of rotations that will satisfy the criteria in your example. A different way of describing that problem is that you’re trying to find a rotation that aligns the normal $\mathbf n$ of the plane defined by $\mathbf a$ and $\mathbf b$ with the $z$-axis. There’s such a rotation for every rotation axis that lies on the angle bisector of $\mathbf n$ and the $z$-axis: there’s one degree of freedom in the solution.
Let’s go through constructing a rotation for your example problem using these ideas. We’ll construct two rotations: $R_1$, which rotates the standard coordinate vectors to the “before” configuration, and $R_2$, which rotates the standard coordinate vectors to the “after” configuration. The required rotation will then be $R = R_2R_1^{-1} = R_2R_1^T$.
Assuming that $\mathbf a$ and $\mathbf b$ aren’t parallel, so that they define a unique plane, we can take $\mathbf n = {\mathbf a\times\mathbf b\over\|\mathbf a\times\mathbf b\|}$. We normalize now because we’re going to need a unit vector anyway, both for the rotation matrix itself and to compute the angle bisector. We now need to pick a rotation axis. The perpendicular bisector of $\mathbf n$ and $\mathbf e_3$ (the unit vector in the direction of the positive $z$-axis) is a plane with normal $\mathbf n-\mathbf e_3$. (If $n_z\lt0$, you might choose to use $-\mathbf e_3$ instead.) We can pick any vector perpendicular to this to define the rotation axis, $\mathbf n\times\mathbf e_3$ for instance, which will produce the minimal-angle rotation. Normalize this vector to get the unit axis vector $\mathbf k$. We now have two of the three vectors we need for the two frames. For the third, take $\mathbf k\times\mathbf n$ and $\mathbf k\times\mathbf e_3$, respectively. We don’t need to normalize these because they’re cross products of perpendicular unit vectors, so are themselves unit vectors. Plugging these into the two matrices, our rotation is therefore $$\begin{bmatrix}\mathbf k & \mathbf e_3 & \mathbf k\times\mathbf e_3\end{bmatrix}\begin{bmatrix}\mathbf k^T \\ \mathbf n^T \\ (\mathbf k\times\mathbf n)^T\end{bmatrix}.$$ It doesn’t really matter what order you plug these vectors into the matrices or which order you use for the cross products as long as the orders are consistent for the two matrices.
Another approach to constructing a rotation is to use the fact that any rotation can be decomposed into a pair of reflections. This can be more convenient than the previous approach because the construction can proceed with fewer decisions. For the problem at hand, the above approach required selecting a rotation axis even if we didn’t really care about the axis and only wanted some rotation that performed the $z$-axis alignment. You could construct such a rotation by reflecting in the angle bisector of $\mathbf n$ and $\mathbf e_3$ (which you already know how to find) and then reflecting in the plane defined by $\mathbf n$ and $\mathbf e_3$ to flip things around the right way. Both of these reflections only require knowing the normals to the mirror planes and don’t require unit vectors, which can save you the cost of computing some square roots. (Reflections also happen to be their own inverses and transposes, which can save some work, too.)