Rotation matrix between two similar cuboids using their upper sides ( and the planes defined by these sides) I have two different images and with them an estimation of two planes ( defined in the same system). I would like to get the rotation matrix, quaternion or euler angles of a surface within this planes.
What I have is:


*

*an image of an object ( let's say a cuboid to simplify things a bit) taken from a fixed point. 

*an estimation of the plane that I got from the measurement of the upper side of the cuboided form.


then the object is moved ( or even replaced by another one), so I would like to see how I would have to rotate the first measured object to get the position of the second measured object. In both planes I have a point relative to the origin (0,0,0), and the normal (unitary). Plane A and plane B are NOT identical, but the relative position of the surface should be close in area and length.
Please note that, despite I'm using the word "object"  I don't really have a 3D representation or definition, but just a couple of points, and the plane. 
My approach was the following one:


*

*Get two points within plane A to define a vector, and normalize it. With this vector I would define the X or Y axis, as the Z axis is already given with the plane's normal. Calculate the cross product of vector "X-axis", normal to compute the vector "Y-axis". In other words, define coordinate system "A" with 3 unitary vectors. $\vec{AV_x},\vec{AV_y},\vec{AV_z}$.

*Do the same with the second plane, plane "B". Get $\vec{BV_x}, \vec{BV_y}, \vec{BV_z}$.

*Move the system B to the origin of system "A". ( By calculating Point 1 in plane A minus Point 1 in plane B). Calculate the new $\vec{AVx_2},\vec{AVy_2}$ and $\vec{AVz_2}$.

*Compute the rotation matrix, to move B to A. For this, I was calculating a 3x3 matrix M where :
$M_{(1,1)} = \vec{AVx_2} \cdot \vec{BVx}$
$M_{(1,2)} = \vec{AVy_2} \cdot \vec{BVx}$
$M_{(1,3)} = \vec{AVz_2} \cdot \vec{BVx}$
$M_{(2,1)} = \vec{AVx_2} \cdot \vec{BVy}$
$M_{(2,2)} = \vec{AVy_2} \cdot \vec{BVy}$
$M_{(2,3)} = \vec{AVz_2} \cdot \vec{BVy}$
$M_{(3,1)} = \vec{AVx_2} \cdot \vec{BV_z}$
$M_{(3,2)} = \vec{AVy_2} \cdot \vec{BV_z}$
$M_{(3,3)} = \vec{AVz_2} \cdot \vec{BV_z}$
Unluckily, my test so far are showing that my results are not ok. Should I try a new approach to calculate this matrix? ( I just would like to know if this could be ok, or if not, why this should never be used by mankind :) ).
 A: It is still unclear to me, but after reading through it a few times, I think this describes it:
Given two oriented pointed planes $A$ and $B$, find the isomorphism that converts one plane to the other. 
Each oriented pointed plane is defined by:


*

*A reference point, $p$ in the plane,

*A unit vector $\hat n$ normal to the plane.

*A unit vector $\hat h$ parallel to the plane (i.e., normal to $\hat n$).


For convenience, define the number $d = \hat n \cdot p$ and the unit vector $\hat v = \hat n \times \hat h$. The plane itself consists of all points $r$ such that $\hat n \cdot r = d$. The vector $\hat h$ determines the orientation within the plane as its horizontal direction, and $\hat v$ as the vertical direction. And $p$ is the origin. Objects associated with plane $A$ will have an $A$ subscript and similarly for $B$.
An isomorphism of pointed oriented planes carries one plane to the other, preserves distances, the reference points, and the orientation. It can be shown that such an isomorphism $F$ must have the form:
$$F(x) = Mx + b'$$
for some linear transformation $M$ with $\det M = 1$ and some constant vector $b'$. To preserve reference points $F(p_B) = Mp_B + b' = p_A$, so we can rewrite $F$ as:
$$F(x) = M(x - p_B) + Mp_B + b' = M(x - p_B) + p_A$$
Since $F$ must also preserve orientation in the planes, $F(\hat h_B + p_B) = \hat h_A + p_A$, or just $M\hat h_B = \hat h_A$, and similarly, $M\hat v_B = \hat v_A$ and $M\hat n_B = \hat n_A$.
If we consider $\hat n$, $\hat h$, $\hat v$ to be column vectors, then we can form a $3\times 3$ matrix $T = \begin{bmatrix} \hat h & \hat v & \hat n\end{bmatrix}$. That is, $\hat h$ is the first column of $T$, $\hat v$ is the second column, and $\hat n$ is the third. Since all three are unit vectors and they are all orthogonal to each other, $T T^T = T^TT = I$. Now, the three relations with $M$ above show that $$MT_B = T_A$$ And therefore,
$$M = MI = M T_BT_B^T = T_AT_B^T$$
Other than window dressing this is similar to what you did, but there are a couple of places you messed up, and another where we differ that may just me a matter of different conventions: In your steps,


*

*Your $\vec V_x, \vec V_y, \vec V_z$ correspond to my $\hat h, -\hat v, \hat n$, respectively (or at least I assume that by "cross-product X-axis, normal" you mean $\vec V_y = \vec V_x \times \vec V_z$, not $\vec V_y = \vec V_z \times \vec V_x$). I made my choice under the assumption that $\hat n$ points toward the viewer, $\hat h$ points to the right, and $\hat v$ points up.

*(Same as 1)

*The second part of this was a mistake. The thing is, $\vec V_x, \vec V_y, \vec V_z$ are vectors here, not points. That is, they are being used to specify directions in space, not locations in it. The operation of translation affects only locations, not directions. None of the vectors $\vec{AV_x},\vec{AV_y},\vec{AV_z}, \vec{BV_x},\vec{BV_y},\vec{BV_z}$ are modified by the translation.

*In my notation (and ignoring the sign difference and transformation from (3)), your matrix is given by $M = T_B^TT_A$, not $T_AT_B^T$ as I have.

