If I take a photo of a rectangular playing card, I can reconstruct the screen/photo-space position of one missing corner which lies outside the photo (outside the camera frustum): assuming simple perspective projection and no lens distortion, I just intersect the two lines along the adjacent edges, since perspective projection preserves lines.

Is it possible to reconstruct the screen-space position of two missing corners -- an entire missing edge -- that fell outside the camera frustum, given:

  • the screen position of the two known corners, with one full edge between them
  • the card dimensions (e.g. 3" x 5" -- probably the aspect ratio is more important than the actual dimensions here?),
  • a guess at whether the full edge is a "long edge" or "short edge", and
  • the angles observed at the two corners, as well as
  • a model of the camera frustum (e.g. horizontal and vertical field of view angles, and/or sensor dimensions and focal point distance -- do we need this model at all)?

Intuition seems to indicate that the angles formed at the known corners would be a good indicator of the general direction of perspective, as well as the "magnitude of the perspective division" (please excuse any incorrect terminology).

It also seems like I could potentially work to try to find the card face normal vector in camera-relative 3D space given this information, and possibly extrapolate from there, but I'm a little stumped.

OpenCV uses a set of four $x, y$ point pairs to solve a system of linear equations in its getPerspectiveTransform method. I've tried supplementing this with various other "known" information here, while leaving half of two point pairs unknown, but I run into missing information or an inability to get from what I have to what I need.

If there's not a single or few possible solutions (sounds like that's the case, see comments), what form would the space of possible solutions take? What additional information could be provided to narrow the solution to a few roots or fewer DOF?

(From the computer vision side of things, this is probably the entirely wrong approach to take to a card recognition problem -- something like a feature extractor+descriptor+matcher is much more likely to be the right decision, based on what I'm reading -- but I'm curious about a possible analytical solution!)

(I've been wandering around searching for writing on rigid bodies and quads under perspective transforms, but have been having a little bit of trouble fitting what I'm reading to this particular problem; any other references/suggestions would be appreciated as well.)

  • 1
    $\begingroup$ There was one answer here saying this is impossible -- too many DOF with a rectangle -- but possible with squares, if you know the horizon. It promised an image, but the answer has since disappeared, sadly. =( If a 1-or-2 DOF solution space is possible, it may get me close enough... $\endgroup$ – leander Feb 10 '15 at 23:13
  • $\begingroup$ It looks as if math.utah.edu/~treiberg/Perspect/Perspect.htm#UsingVP has the basics for doing this with square/grid items; working through to try to see if it can possibly apply to rectangles. $\endgroup$ – leander Feb 11 '15 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.