Find rigid transform from coordinates of the same points in different reference frames

Given two* 3-dimensional points $p_1$ and $p_2$ expressed in different reference frames $A$ and $B$, find the rigid transform (rotation and translation) between frames $A$ and $B$.

The answer to this question is helpful but only covers the case where $A$ and $B$ have the same origin:

Finding a Rotation Transformation from two Coordinate Frames in 3-Space

My idea to use the SVD approach from the above answer with homogeneous coordinates. The hope is to extract the translation from the 4th column of the result, but I'm unsure if this will work as expected. Can anyone convince me that it will or won't work, or suggest a different approach?

*This problem is a part of a computer vision program in a robotics application. I'm trying to find the transform between a 3D camera and the origin of the robot. I can extract up to 4 distinct points in each frame. For Wahba's problem I only need two, but I suspect that to find the rigid transform I'll need 3.