I have the following problem:

A kinect camera capture a 3D surface and save it as a .obj files containing all the positions of the vertices (in the kinect coordinate system).

If I take a picture from the kinect, I can successfully apply that picture on the 3D surface. (see my previous question here)

Now, I want to do the same thing, except the picture will be taken from a different camera (a webcam) placed on the kinect.

As you can see from my previous question, I only needed the camera pose, the X Y Z point and the intrinsic matrix.

I have the intrinsic matrix from the webcam, I still have access to the X Y Z point (thanks to the .obj file), and I have the pose of the kinect.

Since I have the kinect pose, I needed to find a transformation matrix to go from the kinect pose to the webcam pose. (the pose is described by a rotation matrix and a translation vector)

In order to do so, I determined the pose of the webcam and the kinect individually in a common coordinate system (see the answer here for more details), I used millimeters as units because I believe the kinectfusin toolkit uses them as well.

So now, I have two Rotation|Translation matrices, one describing the pose of the kinect and the other describing the pose of the webcam. Btw, I guess I did it right because the translation vectors seem correct when I measure the distance between the cameras (with some errors of some millimeters tho)

After some testing I found that openCV and kinectfusion use different coordinate systems

coordinate systems

So in order for the openCV coordinate system to match the kinectfusion one, I just multiply the Z coordinate of the translation vectors by -1 (Should I do something about the rotation matrices ??)

Then, to get the transformation I applied the following formulas:

relative_translation_vector = webcam_translation_vector - kinect_translation_vector relative_rotation_vector = webcam_rotation_matrix * inverse(kinect_rotation_matrix)

I then multiplied the R|T matrix from the kinect with the R|T matrix representing the relative transformation and applied the following formula (see my previous question here again):


Well...I thought that would be good enough, but unfortunately my texture is not projected on the correct location..so I guess somewhere in the maths I got something wrong...

It's been many days I'm trying to figure out the problem but I can't find what my mistake is. Any help or suggestions will be greatly appreciated ! Thank you


For people who might be interested here is what I did to solve the issue:

  • the pictures taken from the kinect and the cameras had different axes (x axis was flipped). So before doing the calibration make sure to flip the pictures so they all have the same axis. (Don't know if it had any effects tho..but just to be sure)

  • I noticed my webcam had a hardware defect...so I think it created a lot of distortions. I bought a new one and everything worked perfectly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.