# Robustly map rotation matrix to axis-angle

The Wikipedia article for rotation matrix gives the following formula for converting from rotation matrix, $Q$, to axis-angle, $u$ and $\theta$:

\begin{align} x &= Q_{zy} - Q_{yz} \\ y &= Q_{xz} - Q_{zx} \\ z &= Q_{yx} - Q_{xy} \\ r &= \sqrt{x^2 + y^2 + z^2} \\ t &= Q_{xx} + Q_{yy} + Q_{zz} \\ u &= \frac{1}{r}\begin{bmatrix}x \\ y \\ z \end{bmatrix} \\ \theta &= atan2(r, t - 1) \\ \end{align}

The article goes on to say that if the trace is negative then a different approach should be used but it does not go into detail.

A fully robust approach will use different code when t, the trace of the matrix Q, is negative, as with quaternion extraction.

The quoted sentence refers to the earlier portion of the article describing conversion from rotation matrix to quaternion. In that case, the reason to check for a negative trace was that one of the formulas given would produce division by $0$ for a trace of -$1$. That isn't an issue for the formula above. $atan2(y, x)$ produces a signed $arctan(\frac{y}{x})$ but has some special cases, including for when $x = 0$, so there is no risk of division by $0$.

Is a negative trace ever problematic for the formula above? If so, how should a negative trace be handled?

There's a very related question about another Wikipedia article that gives a different formula for this conversion.

• Quick idea: a rotation in three dimensions has exactly one eigenspace, the axis of rotation. I believe there are eigenspace-finding algorithms that are numerically stable for large classes of matrices, and rotation matrices are particularly nice, so you might be able to find an eigenspace algorithm that's stable for all rotation matrices. – Vectornaut Apr 11 '16 at 18:34

Notice that your formula is equivalent to first solving for a unit quaternion representing your rotation, and then extracting the axis-angle from the quaternion. This approach can be made robust by using one of several possible equivalent formulas for the first step, so as to remain numerically robust near rotation angles of 0 and $\pi$: see for instance http://www.ee.ucr.edu/~farrell/AidedNavigation/D_App_Quaternions/Rot2Quat.pdf.
The formula in the question is poorly behaved for very small and very large rotations. The formula divides by $r$ which approaches $0$ as $\theta$ approaches $0$ and as $\theta$ approaches $\pi$. This can be seen in the following expression for r.
$$r = \left| 2sin(\theta) \right|$$
This relationship can be derived similarly to how $Tr(Q) = 2cos(\theta) + 1$ is derived. One version of that derivation is here.
As $r$ approaches $0$, the trace approaches $3$. As $r$ approaches $\pi$, the trace approaches $-1$. This corresponds to the special cases treated in this question.
These instabilities are inherent to the axis-angle representation of rotation. For the identity rotation there is no uniquely valid axis of rotation and for rotations by $pi$ there are two valid axes of rotation. At or very near to these cases, some arbitrary choice of axis must be made.