Finding a specific camera transformation matrix I have the following situation:
- two targets with known coordinates with respect to the "world". They are on a fixed xy plane on a height 0 in the z-direction.
- Both targets have an angle associated with them that specifically points to the origin of the world at (0,0,0)).
- The camera will automatically measure the centroid of the targets in pixel (x,y) coordinates.
- The camera only looks directly down at the targets, but can translate in x,y,z. When I calibrate the transformation, the (x,y,z) position of the camera is known, but after calibration it will be unknown and must be found by looking at the targets.
Question: Is it possible to find and calibrate a transformation matrix that will take a snapshot of 2 target (x,y) pixel locations and their angles and output the real (x,y,z) of the camera centered coordinate frame with respect to the world? If so, how would I do the measurements/math to calibrate it?
All of the camera matrix examples I have seen create a projective version of the real (x,y,z) and then say that it is equivalent within a scaling factor. But in a real situation I need that scaling factor to get the real z position. Getting the height of the camera in z is the dilemma that I am facing. 
Diagram: 
Edit: I read here (http://www.robots.ox.ac.uk/~az/tutorials/cvpr03_part1.pdf) that 4 measurement points define a general camera transformation. But I don't think I understand how they did the math based on reading it. Is it true that if I had 4 targets I could find the position and the orientation of the camera? What about if I had 3 targets and all of their angles? 
 A: Since the plane in which the targets lie is parallel to the image plane, the homography between them is a simple affine transformation, specifically, a uniform scaling with translation (with a rotation as well if the camera is allowed to rotate about its axis.) It’s easy enough to construct a matrix for this transformation. Let the camera’s position in world coordinates be $\tilde{\mathbf C}=(x_c,y_c,z_c)$ with $z_c\gt0$ and the camera’s effective focal distance be $f$, so that the image plane is $z=c_z-f$. The corresponding projection matrix is $$\mathcal P=\begin{bmatrix}f&0&-x_c&(z_c-f)x_c\\0&f&-y_c&(z_c-f)y_c\\0&0&-(z_c-f)&(z_c-f)z_c\\0&0&-1&z_c\end{bmatrix}.$$ If we take the point directly under the camera as the image coordinate origin, then we get the homography $$\mathcal H=\begin{bmatrix}\frac f{z_c}&0&-\frac f{z_c}x_c\\0&\frac f{z_c}&-\frac f{z_c}y_c \\ 0&0&1 \end{bmatrix} = \begin{bmatrix}\frac f{z_c}&0&0\\0&\frac f{z_c}&0 \\ 0&0&1 \end{bmatrix}\begin{bmatrix}1&0&-x_c\\0&1&-y_c\\0&0&1\end{bmatrix},$$ i.e., a translation to the camera center followed by scaling by a factor of $f/z_c$. The camera’s intrinsic matrix will compose this with an additional affine transformation. For now, we’ll consider the simple case of square pixels with no skew, so that the resulting transformation is still effectively a combination of translation and uniform scaling. The inverse of this matrix separates the scale factor from the translation, allowing us to read both directly from the matrix: $$\mathcal H^{-1}=\begin{bmatrix}\frac{z_c}f&0&x_c\\0&\frac{z_c}f&y_c \\ 0&0&1 \end{bmatrix}.$$  
How you recover the camera’s world-coordinate position from the measured locations of the targets will depend on what you know about the camera and the targets. Obviously, since you know the direction of the origin from each target, computing the image of the world origin is a simple matter of finding the intersection of a pair of lines. This gives you three noncolinear points in the image $\mathbf q_0$, $\mathbf q_1$ and $\mathbf q_2$—the world origin and the two targets. If you know the ($x$-$y$ plane) world coordinates $\mathbf p_1$ and $\mathbf p_2$ of the targets, then you can recover $\mathcal H^{-1}$. It is simply $$\begin{bmatrix}\mathbf p_0&\mathbf p_1&\mathbf p_2\end{bmatrix}\begin{bmatrix}\mathbf q_0&\mathbf q_1&\mathbf q_2\end{bmatrix}^{-1}.$$ From this you can recover $x_c$, $y_c$ and the ratio $z_c/f$. Unfortunately, without knowing the effective focal distance $f$, you can’t recover the camera’s $z$-coordinate. If you can identify a pair of lines in the image that correspond to orthogonal lines in the source plane, there’s a way to recover $f$, but otherwise you’ll have to get this information elsewhere.  
If you don’t know the world coordinates of the targets, but do know their actual sizes, you can use that to recover the scale factor, and with that in hand you can then work out $x_c$ and $y_c$, but that will require more data from the image than just the centroids of the targets. If the camera has non-square pixels and/or has a skew, the source-to-image plane transformation matrix will be of the form $$\begin{bmatrix}s_x&\alpha&t_x\\0&s_y&t_y\\0&0&1\end{bmatrix}$$ with inverse $$\begin{bmatrix}\frac1{s_x}&-{\alpha\over s_xs_y}&-\left({1\over s_x}t_x-{\alpha\over s_xs_y}t_y\right)\\0&\frac1{s_y}&-{1\over s_y}t_y\\0&0&1\end{bmatrix}.$$ As before, finding the $x$- and $y$-coordinates of the camera is simple—they are the last column of the inverse matrix—but teasing out the $z$-coordinate will require more information about the camera.
