I have a RGB image, and for each pixel in the image I also have its real world coordinate. I also have the location (real world coordinate) yaw, pitch and roll of the camera. I am trying to produce a depth map but my maths is letting me down.

Basically, I have a array points (world coordinate frame) and a camera location (also world coordinate frame) and yaw and pitch of camera. I want to find the distance of each point from the plane containing the camera (focal plane). From my searching, trying to understand pinhole camera model and coordinate frames I have managed to confuse my self.

Assuming the math below is correct, the distance I seek is just the dot product of the point and the up vector. But I can't seem to calculate the up vector (probably some thing simple).

The focal plane itself is likely not aligned with the x,y,z axes of the world. However, if we can calculate the direction into the scene (orthogonal to the plane) and the up vector (incident to the focal plane). We can compute the orthogonal distance to the point (from the focal plane) by noting that:

  cos(t) = p.q/(|p||q|) 
       where p is the position vector of the point of interest, 
             q is the unit direction vector into the scene, 
             p.q is the dot product, 
             |p| is magnitude of p and 
             t is the angle between the vectors p and q.

 But c/h = cos(t) 
     where h = |p| and 
           c is the orthogonal distance from the focal plane.
       c = h*cos(t)
       c = h * p.q/(|p||q|)
       c = p.q/|q|   (since h = |p|)
       c = p.q       (since |q| = 1, recall q is unit focal vector)

Define pitch=0 as horizontal (z=0) and yaw as counter-clockwise from the x axis, then the direction vector will be:

x = cos(yaw)*cos(pitch)
y = sin(yaw)*cos(pitch)
z = sin(pitch)

Assuming the math is correct, how can I calculate the unit "up" vector orthogonal to the direction into the scene?

If the math is incorrect, how should I calculate this distance?

Is their an easier/better/more robust way than I am trying?

EDIT: Fixed some terminology based on solution provided below.

  • $\begingroup$ If you have or can reconstruct the $4\times3$ projection matrix $P=\left[M\mid \mathbf p_4\right]$, then the last row of $M$, $\mathbf m^3$ is a vector parallel to the camera axis, and $\det(M)\mathbf m^3$ points toward the front of the camera. $\endgroup$ – amd Jun 2 '18 at 18:39

I'm not sure what you mean by “focal vector” in your computation. Is it supposed to be a vector within or orthogonal to the focal plane?

In order for that computation to make sense, you'll need q to be the direction vector, i.e. the unit vector of the direction in which the camera is pointed, orthogonal to the focal plane. You don't need any “up” direction for this. The dot product will be maximal (i.e. equal to |p| if p lies in the same direction as q, and will be zero if p is orthogonal to q, i.e. in the focal plane. The vector p should obviously be computed relative to the camera location.

For future reference, in case you or someone reading this might indeed need that “up” vector: you can compute that as follows:

x = -cos(yaw)*sin(pitch)
y = -sin(yaw)*sin(pitch)
z = cos(pitch)

You can think of this as follows: take the camera with pitch=0. Then “up” will be the z direction. When you change pitch, the z component will decrease, and the x and y components will increase. More specifically, when you increase pitch, then the x any y coordinates of the up vector will point away from the direction of your camera. Hence the minuses.

  • $\begingroup$ Thanks, you have answered my question, even thouhgh as you rightly point out I don't really need the "up" vector. Sorry for the confusing terminology. By "focal vector" I meant the vector in the direction the camera is pointed, i.e. the direction vector. Obviously I had confused my self sufficiently and was thinking it had to be the "up" vector, which wouldn't work. Thanks again. $\endgroup$ – Michael Apr 30 '13 at 7:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.