I have a 3x4 projection matrix for the camera and a set of 2d coordinates of the desired object in the image. How do I find the 3d coordinates of the object?

This is how the matrix looks like:

$$ \left( \begin{array} .22.989418 & 4.628043 & 0.044168 & 49.197014 \\ 0.534536 & -2.27624 & -34.054806 & 90.715378 \\ -0.175571 & 0.980961 & -0.083006 & 106.175468 \\ \end{array} \right) $$

[521.46, 224.54] are the 2d coordinates in the image.

  • $\begingroup$ Are these the correct equations? homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/EPSRC_SSAZ/… $\endgroup$ – Matti P. Jan 14 at 12:03
  • $\begingroup$ I looked at that earlier but couldn't it out to use to my case. $\endgroup$ – sspatole Jan 14 at 12:28
  • $\begingroup$ Without more information, you can’t. Each point in the image back-projects to a ray in the scene. $\endgroup$ – amd Jan 14 at 20:37
  • $\begingroup$ @amd what more information would I require? $\endgroup$ – sspatole Jan 14 at 20:40
  • $\begingroup$ Basically, anything that would let you establish the object location’s depth in the image. It could be known locations of other objects with know relations to it, the original size/shape/orientation of the object, &c. $\endgroup$ – amd Jan 14 at 20:44

In short, with only the information presented in your question, you can’t.

The basic problem is that depth information is lost: a point in the image corresponds to a ray emanating from the camera center in the scene. In this case, your object could be anywhere along the ray $(18.785,-104.035,9.9124)+\mu(21.815,4.3702,-6.5432)$. You can check for yourself that your camera matrix maps this entire ray to $(521.46,224.54)$.

Having the camera matrix gets you a lot, but not this missing depth information. There are many ways to extract size and position information from an image, but they all require some known reference to get things started. For example, if you know that this object is at ground level and know the equation of the ground plane in the scene, you can find the intersection of this plane with the back-projection ray. Assuming that the object is more than just a single pixel in the image, if you know something about its size/shape/orientation, you could use that to recover its depth in the scene. And so on.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.