Relative camera matrix (pose) from global camera matrixes I have a list of camera poses from a given ground truth.
Each pose is given in the form of a quaternion and a translation, from some arbitrary world origin.
Each pose can be assembled into a 4x4 camera matrix of the form :
$ P = \begin{bmatrix} R & t \\ \mathbf{0}^t & 1 \end{bmatrix} $
Now given P1, and P2 with their respective R1|t1 and R2|t2, I want to compute $P_{relative}$ between the two.
Is there a way to do that directly with P1 and P2, or do I need to compute the relative rotation and translation separately ?
Thank you!
 A: With cameras $C_1$ and $C_2$ with respective camera matrices $P_1^{\{W\}} = \begin{bmatrix} R_1 & t_1 \\ \mathbf{0} & 1 \end{bmatrix}$ and $P_2^{\{W\}} = \begin{bmatrix} R_2 & t_2 \\ \mathbf{0} & 1 \end{bmatrix}$, where $W$ denotes the world frame, we want to find the transformation matrix $P_1^{\{2\}}$ that is the transformation from $C_1$ to $C_2$. You can just use $P_1^{\{W\}}$ and $P_2^{\{W\}}$ to find this, since you know they are both given in the same frame.  The basic process is to transform from $C_1$ to $W$ to $C_2$.  
Step 1: 
Given a point $q^{\{1\}}$ in $C_1$, the the world coordinate  is given by $q^{\{W\}} = t_1 + R_1 q^{\{1\}}$ 
Step 2: 
Given a point $q^{\{W\}}$ in $W$, the $C_2$ coordinate is given by $q^{\{2\}} = R_2^{-1} (q^{\{W\}} - t_2)$
Step 3: 
Combine steps 1 and 2.  You have 
$$ q^{\{2\}} = R_2^{-1} (q^{\{W\}} - t_2) $$
$$ q^{\{2\}} = R_2^{-1} ((t_1 + R_1 q^{\{1\}}) - t_2) $$
$$ q^{\{2\}} = R_2^{-1} (R_1 q^{\{1\}} + t_1 - t_2) $$
$$ q^{\{2\}} = R_2^{-1} R_1 q^{\{1\}} + R_2^{-1} (t1-t2) $$
which you can write as 
$$ q^{\{2\}} = P_1^{\{2\}} q^{\{1\}} $$
where 
$$
 P_1^{\{2\}} = \begin{bmatrix} R_2^{-1} R_1 & R_2^{-1} (t_1 - t_2) \\ \mathbf{0} & 1\end{bmatrix} 
$$
If you'd like to simplify with notation a bit, and knowing that since $R_2$ is orthonormal that $R_2^{-1} = R_2^T$, you can write
$$
 P_1^{\{2\}} = \begin{bmatrix} R_2^{T} R_1 & t_{12} \\ \mathbf{0} & 1\end{bmatrix} 
$$
where $t_{12} = t_1^{\{2\}} - t_2^{\{2\}}$.
