# How to find 3d world coordinates from projection matrix?

I have a 3x4 projection matrix for the camera and a set of 2d coordinates of the desired object in the image. How do I find the 3d coordinates of the object?

This is how the matrix looks like:

$$\left( \begin{array} .22.989418 & 4.628043 & 0.044168 & 49.197014 \\ 0.534536 & -2.27624 & -34.054806 & 90.715378 \\ -0.175571 & 0.980961 & -0.083006 & 106.175468 \\ \end{array} \right)$$

[521.46, 224.54] are the 2d coordinates in the image.

• Are these the correct equations? homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/EPSRC_SSAZ/… – Matti P. Jan 14 at 12:03
• I looked at that earlier but couldn't it out to use to my case. – sspatole Jan 14 at 12:28
• Without more information, you can’t. Each point in the image back-projects to a ray in the scene. – amd Jan 14 at 20:37
• @amd what more information would I require? – sspatole Jan 14 at 20:40
• Basically, anything that would let you establish the object location’s depth in the image. It could be known locations of other objects with know relations to it, the original size/shape/orientation of the object, &c. – amd Jan 14 at 20:44

The basic problem is that depth information is lost: a point in the image corresponds to a ray emanating from the camera center in the scene. In this case, your object could be anywhere along the ray $$(18.785,-104.035,9.9124)+\mu(21.815,4.3702,-6.5432)$$. You can check for yourself that your camera matrix maps this entire ray to $$(521.46,224.54)$$.