# Find correspondences between a set of ellipsoids and a set of ellipses

I am having the following problem. Imagine that I have set of quadrics, i.e. Ellipsoids in the 3d space and a set of conics, i.e. ellipses on an image plane. For each of the sets I have all the information that describe these two sets as 3d and 2d gaussian distributions respectively. Thus, the mean, μ, and the covariance matrices Σ.

My question is, based on this information is there an approach that I could possibly obtain some good correspondences so that I can match an ellipsoid with an ellipse. I am trying to figure out whether geometrically or statistically is possible to get something.

The inspiration comes from this work: Perpective-1 Ellipsoid where it is possible to recover the camera pose based on one Ellipsoid to Ellipse correspondence but in the paper the correspondence is considered as known.

Apparently there are some distance metrics which can be used to obtain a distance between gaussian distributions but they demand them to be of the same size.

Then an other idea was to project the quadric 3x3 covariance as the Jacobian of the projection function as also described here and check if I can then use this information for matching with the 2x2 covariance matrices of my ellipses set. I know that the Jacobian is gonna be an approximation but I believe that it could be good enough for the task I need it. I am not quite sure though how good this could work, and whether it could be possibly set as an optimization problem(?).

Also in the aforementioned paper there is also the concept of projection and back-projection cones, B and B', and their alignment between the ellipsoid and its corresponding ellipse which is done through a scaling factor, B = σB', and which could be quite useful. The issue here though comes from the fact that I do not have the initial position of the camera.

The whole concept is quite new to me, thus any feedback would be appreciated.