For perspective projection with given camera matrices and rotation and translation we can compute the 2D pixel coordinate of a 3D point.
using the projection matrix,
$$ P = K [R | t] $$
where $K$ is intrinsic camera matrix, $R$ is rotation $t$ is translation. The projection is simple matrix multiplication $x = P X $. Zisserman's book, pg. 161 suggests using $3 \times 4$ projection matrix and taking pseudoinverse. Then one would compute $X$ which defined up to scale which can then be interpreted as the ray starting from camera center going to infinity. I quickly coded this up, I took $Z$ as depth, so I translated the camera in $Y$ direction (up 1 meter), and after retrieving $X$ flipped $Y,Z$ for plotting (most projective geom. math seems to be built to make $Z$ depth),
K = [[ 282.363047, 0., 166.21515189], [ 0., 280.10715905, 108.05494375], [ 0., 0., 1. ]] K = np.array(K) R = np.eye(3) t = np.array([,,]) P = K.dot(np.hstack((R,t))) import scipy.linalg as lin x = np.array([300,300,1]) X = np.dot(lin.pinv(P),x) X = X / X from mpl_toolkits.mplot3d import Axes3D w = 20 f = plt.figure() XX = X[:]; XX = X; XX = X ax = f.gca(projection='3d') ax.quiver(0, 0, 1., XX[:3], XX[:3], XX[:3],color='red') ax.set_xlim(0,10);ax.set_ylim(0,10);ax.set_zlim(0,10) ax.quiver(0., 0., 1., 0, 5., 0.,color='blue') ax.set_xlabel("X") ax.set_ylabel("Y") ax.set_zlabel("Z") ax.set_title(str(x)+","+str(x)) ax.set_xlim(-w,w);ax.set_ylim(-w,w);ax.set_zlim(-w,w) ax.view_init(elev=29, azim=-30) fout = 'test_%s_01.png' % (str(x)+str(x)) plt.savefig(fout) ax.view_init(elev=29, azim=-60) fout = 'test_%s_02.png' % (str(x)+str(x)) plt.savefig(fout)
These images below are the result (blue arrow shows the normal vector perpendicular to the image plane, the images demonstrate all x=10,300 y=10,300 combinations):
I give the camera/ray plot for each pixel from two different angles.
Do these results look sensible? 10,10 and 200,200 looked odd, I played around with signs a little bit, if I translate up using negative -1, and using -Z after X calc., things improve somewhat?
t = np.array([,[-1],]) .. XX = X[:]; XX = X; XX = -X
I do not know why that is.