How do I compare the (quaternion) orientation of two objects and specify different tolerances along each local axis? I have two objects in 3D space with orientations represented by quaternions (but I can also convert to Euler angles if necessary) and I'd like to find out if they are oriented in more or less the same direction.
I'm a bit out of my depth for this kind of math, so I'm not sure what terms I'm looking for. Even being able to name the operations I need would help a lot.
The simplest case is if one of the objects is axis aligned: 


*

*Object A has zero roll, zero pitch, and zero yaw in world space.

*Object B is "close enough" to Object A if it has:


*

*Roll less than 45 and more than -45

*Pitch less than 90 and more than 0

*Yaw less than 180 and more than 0.



(All the numbers are just example inputs to the math I'm looking for).
Another way I'm thinking about it is, I essentially want to be comparing the two up vectors of each object to each other (as well as the forward and right vectors). But that means I have to specify the up vector tolerance in terms of the... right vector? Like, is there a way to get Object B's pitch, roll, and yaw in the local space of Object A?
Two problems I can't figure out:


*

*How do I account for Object A being at an arbitrary rotation like roll=55, pitch=180, yaw=10? 

*There is more than one combination of pitch/yaw/roll for every orientation, so I can't just compare the RollA to RollB in world space. Similarly, can I just take to absolute value of the quarternion to account for q and -q being the same orientation?


If you have a solution that doesn't incorporate both min and max tolerances for each axis, and just has an absolute tolerance, I'd be interested in hearing that solution, too.
EDIT: Here's more details of my use case:
In my application Object B is a motion tracked human hand, and Object A is a motion tracked head. Their orientations are stored as quaternions. I want to show a graphic when you hold your hand in front of you. You can hold your hand flat and facing up (like a waiter carrying a tray), or vertically in front of your face (like you're playing peekaboo), or anything in between.
However, if you tilt your hand more than 45 degrees to the side (as if you were reaching out to shake someone's hand) I don't display the graphic.
That means I need a tolerance of 90 degrees of hand pitch, and only 45 degrees of hand roll. Where hand pitch and hand roll are relative to the head (so you can still see the graphic if you're laying on your back or hanging upside down).
 A: If I understand well, you have two objects obj1 and obj2 with associated unit quaternions $q_1$ and $q_2$ which define their "orientation", so that by comparing their quaternions you can tell if they are heading to similar directions.
Algebraicly speaking, there are infinitely many quaternions that produce the same rotation, however numerically we can restrict ourselves to two cases where $q$ and $-q$ represent the same rotation but with opposite sense.
The first measure of similarity of quaternions is their product.
$d = q_1 q_2^*$
The reason is that if $q_1$ and $q_2$ are exactly equal, that product is equal to $d = 1$. Looking at how far is the the scalar part of quaternion $d$ from $1$ is a measure of disimilarity. You should care of the $d = -1$ case too, which mean that they differ only in sign.
Other meaningful measure is direct comparison of thier logarithms. The logarithm of a quaternion is basically the axis of rotation multiplied by the angle of rotation.
LOGARITHM
A quaternion $q$ that rotates an object around an axis $b$ by $\theta$ radians, can be represented as $q = s + v$ where $s = \cos(\theta/2)$ and $v = - \sin(\theta/2) b$ and $\|b\| = 1$.
The logarithm of $\log(q) = \theta b$:
Where:
$b = v / \|v\|$
$\theta = 2  \tan^{-1}(\|v\|/s)$
Or
$\theta = 2  \tan^{-1}(\sin(\theta/2)/\cos(\theta/2))$
You can compare the parallelness of axis of rotation using the dot product and if they are mostly parallel the angle difference would be meaningful.
The (scalar part of) commutator product $q_1 q_2 - q_2 q_1$ is zero if $q_1$ and $q_2$ commute, two quaternions commute only if their axis of rotation is the same. That test can also be applied to quaternion logarithms.
You can even compare the euclidean distance of quaternions, i.e., treat them as 4D vectors and compute how much $\|q_1 - q_2\|$ differ from $0$. It works also with logarithms. It depends on applications wether this is meaningful or not, since quaternions with negative sign will be consider different by this test as you will be comparing direct similarity of their coordinates.
