# Resolving the up-vector of an image through multiple transforms

First up, a disclaimer. I'm an engineer not a mathematician, so my approach thus-far may be a little ham-fisted. I have however reached a point in this problem where my usual engineering references aren't helping, and since I'm phrasing this as a vector and co-ordinate frame problem I figured I'd try asking here.

The problem is as follows. I have a pan-tilt system which points a camera in a specified direction (azimuth and elevation) relative to it's body frame. We define the pointing vector of the camera to be a unit vector in the body frame pointing along the desired azimuth and elevation,. The up vector for the image is therefore a vector perpendicular to the pointing vector which is co-planar with the Z (up) axis of the body frame.

We have a intertial unit on the pan-tilt system which gives us the roll-pitch-yaw rotations between the body frame and a global reference frame. The main task is to, when given an azimuth-elevation pointing angle command in the global frame, calculate the required Az and El for the pan-tilt unit in the body frame.

This enables us to keep the camera pointing in the same global direction as the vehicle carrying the sensor pitches, rolls and yaws underneath us, and is working just fine. The problem is that we are not yet accounting for any rotation of the unit about the pointing vector, and therefore the image from the camera appears to roll as the platform moves.

My approach to solve this problem is to attempt the following:

1. Translate the global up-vector into the body frame of the pan-tilt unit (trivial).
2. Find the projection of that translated global up-vector in the image plane (the perpendicular plane to the pointing vector)
3. Find the angle between the image up-vector and the projected global up-vector.
4. De-rotate the image in software by this angle. (trivial)

My problems are two-fold. Firstly, I can't figure out how to find the projection of the translated global up vector in the image plane, or even whether such a thing is meaningful without some further definition of a co-ordinate frame on that image plane. Secondly, this algorithm needs to be written to be extremely computationally efficient. I need to be able to calculate this parameter on a lightweight microcontroller at a rate upwards of 200Hz.

So, if there is some standard solution which I'm missing, please point me in the direction of a good reference so I can try and get my head around what is actually going on. Alternatively, if anyone can spot a simpler approach to this one then I'd absolutely love to hear it.

Thanks for the help from a seriously confused engineer.....

-

$$\mathbf{a} - (\mathbf{a}\cdot\mathbf{\hat b})\mathbf{\hat b}$$
where $\mathbf{a}$ is the up vector and $\mathbf{\hat b}$ is the (normalized) pointing vector.