Suppose you're given two planes
$ n_1 \cdot r = d_1 $
$ n_2 \cdot r = d_2 $
where $n_1$ and $n_2$ are unit vectors (known). $d_1$ and $d_2$ are known scalars. I want to rotate the first plane into the second plane with the following condition. Points $P_1$ and $Q_1$ are on the first plane, and points $P_2$ and $Q_2$ are on the second plane, such that $\overline{P_1 Q_1} = \overline{P_2 Q_2} $. I want to find the rotation(s) that will rotate the first plane into the second plane while mapping $P_1$ to $P_2$ and $Q_1$ to $Q_2$. How can I do this ? Your help is much appreciated.
My attempt:
We have two vectors in space that we need to relate by a $3D$ rotation matrix. We have
$ n_2 = R \ n_1 $
and
$ u_2 = R \ u_1 $
where the vectors are column vectors (algebraically). $R$ is the $3 \times 3$ rotation matrix, $u_1 = P_1 Q_1$, $u_2 = P_2 Q_2$. I then formed the cross product of the respective pairs of vectors (plane $1$ and plane $2$) and related them by the same rotation matrix. This is possible because it is a fact that
$ R ( \vec{a} \times \vec{b} ) = ( R \vec{a} ) \times ( R \vec{ b} ) $
Hence, we can now write
$ \begin{bmatrix} n_2 && u_2 && n_2 \times u_2 \end{bmatrix} = R \begin{bmatrix} n_1 && u_1 && n_1 \times u_1 \end{bmatrix} $
So that matrix $R$ can be determined easily as follows
$R = \begin{bmatrix} n_2 && u_2 && n_2 \times u_2 \end{bmatrix} \begin{bmatrix} n_1 && u_1 && n_1 \times u_1 \end{bmatrix}^{-1} $
The only thing left is to locate the center of rotation, and this can determined from the fact that
$ P_2 = C + R (P_1 - C) $
This simplifies to,
$ P_2 = R P_1 + (I - R) C $
which is a linear system in the unknown $C$. Now it is a well-known fact that $(I - R)$ is rank deficient, so $C$ will either belong to a line (which is the axis of rotation), or there will be no solution for $C$. This depends on $P_2$.
Edit:
This last equation suggests a different view at the situation. We have the vectors $n_1$, $n_2$, $P_1 Q_1$ and $P_2 Q_2$. These four vectors will generate a specific $R$ (note that $n_2$ or $n_1$ can be replaced with their negative, resulting in a different $R$). The above equation, is
$ P_2 = R P_1 + (I - R) C $
Therefore,
$ P_2 - P_1 = (I - R) (C - P_1) $
Hence, this equation, will have a solution for $C$ if and only if $P_2 - P_1$ is in the column space of $(I- R)$. If the column space is
$ p = t_1 V_1 + t_2 V_2 $
where we can assume that $V_1$ and $V_2$ are unit and orthogonal to each other.
$V_1, V_2$ can be obtained by $QR$ factorization of $(I - R)$.
i.e. $I - R = R_1 G $
where $G_{33} = 0 $
Then $V_1$ and $V_2$ are the first two columns of $R_1$.
Then we must have
$P_2 = P_1 + t_1 V_1 + t_2 V_2 $
we also know that
$ n_2 \cdot P_2 = d_2 $
This restricts $P_2$ to lie on a straight line. If $P_2$ is on this line, then we can solve for $(C - P_1)$ because then
$ R_1 \begin{bmatrix} t_1 \\ t_2 \\ 0 \end{bmatrix} = R_1 G (C - P_1) $
Then
$ \begin{bmatrix} t_1 \\ t_2 \end{bmatrix} = G' (C - P_1) $
where $G'$ is obtained from $G$ by deleting the last row of $G$ which is a row of zeros.
Now, $G'$ has a column rank of $2$, so this $2 \times 3$ system always has a particular solution. And in addition, the homogenous solution is given by the null space of $G'$ (which is one-dimensional).
Therefore, we now have that the center of rotation $C$ lies on a straight line, which is the axis of rotation.
In summary, to obtain the required rotation, I've expressed the rotation matrix in terms of the four vectors $n_1, n_2, P_1 Q_1 , P_2 Q_2 $, then derived the condition for the existence of this rotation, namely that $P_2 - P_1$ lies in the column space of $(I - R)$.