Let's say I have data in one coordinate system. It is somehow rotated against the real world coordinate system. I have a tracker device and its rotation matrix in the mentioned coordinate system. What I want is to transform every measurement from the unknown coordinate system to the real world one.
To do that I'm using a calibration process: I put the device on an even plane, so I know that its roll and pitch angles are zero in the real world coordinate system, then I take a measurement from the device (rotation matrix), make inverse of this matrix and then I use this inverse matrix to multiply every other received rotation matrix to transform it to the real world coordinate system.
Is this process allright? The problem I'm facing is that when I rotate with the device in a yaw direction, also roll and pitch are slightly changing and I don't know if its due to the sensor accuracy or the described process isn't OK. Thanks for any help or suggestions.