Camera Calibration - Calculate Rotation and Translation I have a calibration problem between two cameras, my setup contains two static cameras. Both cameras capture the same scene but from a different viewpoint.
I estimate the individual cameras poses using PnP (Perspective n Point) with the 6D position (rotation and translation (6 degress of freedom)) of objects in the image. However I need to calculate the RT (rotation and translation) between the two cameras. How can this be approached?
EDIT:
if I use PnP, e.g. solvePnP from OpenCV
It returns -> retval, rvec, tvec
so for Cam1, I would have the rotation and translation (R1, t1)
for Cam2 as well (R2, t2)
so could I calculate it like this
R_relation = R1^-1 * R2

t_relation = R1^-1 * (t2-t1)

so: R2 = R1 * R_relation
and t2 = R1 * t_relation + t1
I am pretty new in this field, so thanks in advance :)

 A: In a recent post I've derived the image coordinates $p$ of a camera that has a general center position $d$ and  orientation specified by a rotation matrix $R$ relative the world frame.
The result is that
$ p = \dfrac{f R^T (r_1 - d) } { (R k)^T (r_1 - d) } $
where $r_1$ is a point specified in the world coordinate system, and $k = [0, 0, 1]^T $
The image point is $p = [P_x, P_y, f ] $
Now, the columns of matrix $R$ are unit vectors that are mutually orthogonal, that is,
$R = [u_1, u_2, u_3] $
Only three parameters are needed to specify $R$, and an additional three parameters are needed to specify the location of the origin of the camera reference frame $d$, and in addition we can also assume that the focal distance f is unknown. This makes the total number of unknowns for each camera equal to $7$.  The above equation can be broken down into two equations:
$P_x ( u_3^T (r_1 - d) ) = f u_1^T (r_1 - d) $
$P_y ( u_3^T (r_1 - d) ) = f u_2^T (r_1 - d) $
Hence for each $r_1$ we can write two scalar equations, and therefore, we need $4$ points to write sufficient number of equations to solve for all the unknowns.
To generate the matrix for our numerical solver, we take $u_3$ to have the form
$u_3 = [ \sin \theta \cos \phi, \sin \theta \sin \phi, \cos \theta ]^T $
Next step is to define two canonical vectors that are orthogonal to $u_3$ which I will call $w_1$ and $w_2$ and they are defined as follows
$w_1 = [\cos \theta \cos \phi, \cos \theta \sin \phi , -\sin \theta ]^T $
$w_2 = [-sin \phi, \cos \phi , 0 ]^T $
Then we can take $u_1$ and $u_2$ to be a rotation of $w_1$ and $w_2$ as follows
$ u_1 = \cos \psi w_1 + \sin \psi w_2 $
$u_2 = - \sin \psi w_1 + \cos \psi w_2 $
This completes the specification of matrix $R$ using the three parameters $\theta, \phi, \psi $.
Using a numerical method like Newton-Raphson multivariate method, we can solve for the 7 unknowns, which are $\theta, \phi, \psi, d_x, d_y, d_z$, and $f $.  I have verified with an example and a computer program that this identification works.
This has to be repeated for the second camera, and we end up having two coordinate frames $(d_1, R_1)$ for the first camera and $(d_2, R_2)$ for the second camera.  These two frames can be easily related as follows.  Given a world point $r$ , then
$ r = R_1 p_1 + d_1 = R_2 p_2 + d_2 $
Hence,
$ p_2 = R_2^T (R_1 p_1 + d_1 - d_2 ) $
which simplifies to,
$p_2 = (R_2^T R_1) p_1 + R_2^T (d_1 - d_2) = R p_1 + d $
where $R = (R_2^T R_1) $ and $ d = R_2^T (d_1 - d_2) $
