I am using a camera mounted on a pole to detect objects. This equation below, from OpenCV, gives an equation for finding pixel coordinates from 3D coordinates.

image_pixels_in_2D (3 x 1) = [intrisix_matrix (3 x 3)] . [ extrinsic matrix (3 x 4)] . [ world coord (4 x1)]

For example, using the values that I have

[x, y, 1] T =

           [[1.62101313e+03 0.00000000e+00 9.60000000e+02]
           [0.00000000e+00 1.62025316e+03 1.28000000e+03] 
           [0.00000000e+00 0.00000000e+00 1.00000000e+00]]
                       dot product with 
   [[-3.46094253e-01  9.21477975e-01 -1.76343727e-01  1.59227339e+04]
    [ 6.37135469e-01  9.28747529e-02 -7.65135723e-01 -3.60030568e+05]
    [-6.88677836e-01 -3.77163920e-01 -6.19249720e-01  4.40335632e+05]]
                       dot product with
                     [538000, 180023, 6]

Sorry for the formatting. My coordinates are non-homogenous ( i presumed that it meant that z-axis is non 0). I derived the extrinsic matrix using this non-homogenous data with the tool solvePnP in opencv.

cv2.solvePnP(world_coord_pts, image_pixel_pts, intrMatrix, distCoeffs)

where distcoeff is an empty array. Using some initial points I get my camera matrix, which is =

            [[-1.22215405e+03  1.13165054e+03 -8.80335229e+02  4.48533168e+08]
             [ 1.50813130e+02 -3.32289205e+02 -2.03235322e+03 -1.97110575e+07]
             [-6.88677836e-01 -3.77163920e-01 -6.19249720e-01  4.40335632e+05]]

I wish to do the reverse. The altitude of the location will always be known ( say 6m for this point, maybe 13 m for another point). I read other threads, where it is suggested that the equation

X ~ ( μ ) = M − 1 ( μ x − p 4 ) .

where it is my assumption that my camera matrix needs to be decomposed into M and p4, where M is the first 3x3 matrix of camera matrix =

            [[-1.22215405e+03  1.13165054e+03 -8.80335229e+02]
             [ 1.50813130e+02 -3.32289205e+02 -2.03235322e+03]
             [-6.88677836e-01 -3.77163920e-01 -6.19249720e-01]]

and p4 is the last column of the camera matrix so, it should be


and x would be my pixel coordinates say [30, 900, 1] corresponding to [538000, 180023, 6]

So the first question is what is μ on right hand side. How do I compute it or find it? Do I find it with known 3D and 2D points and then use the result to find the subsequent 3D points for all other pixel points.

Second question , what is μ on the left hand side?

Third question - What does tilde (~) mean on top for capital x (X) in the equation?

Can someone help compute μ given the values above, please.

Thank you. Any help in deciphering this method would be great. I am writing my code in python.

  • $\begingroup$ The other link with the similar question is here math.stackexchange.com/questions/3577395/… But my problem is I was not able to completely understand the method. That is why shared the values, so if someone could just point me in the right direction. Thank you. $\endgroup$
    – user25194
    Commented Apr 8, 2021 at 16:57
  • $\begingroup$ This is not possible in general. Two points in 3D might be mapped to the same pixel. $\endgroup$
    – K.defaoite
    Commented Apr 8, 2021 at 17:02
  • $\begingroup$ @K.defaoite I was warned that that might be the case but the solutions shared in the links seems to have worked for many. Any help understanding the equation will be great. thanks $\endgroup$
    – user25194
    Commented Apr 9, 2021 at 8:33


You must log in to answer this question.

Browse other questions tagged .