3D rotation around arbitrary axis

I have a 3D rotation matrix, R which is a combination of rotations around x-axis , y-axis and z-axis. I know how to calculate n(the arbitrary axis around which a point rotated about theta angle and this rotation is equal to rotating that point using the 3D matrix above) i.e. by finding the eigenvector of R corresponding to the eigenvalue equal to 1. How can I calculate theta, the angle to rotate about the arbitrary axis?

If you know the axis of rotation $A=(a,b,c)$, then you can find a vector orthogonal to this one. For example, if $a \ne 0$, $b \ne 0$, then $V=(b,-a,0)$ is such a vector. Or, if one component is $0$--say, for example $c=0$, then $V=(0,0,1)$ is orthogonal. And that covers all cases.
A third vector in a right-handed triple is found using the vector cross product: $W=A\times V$. Normalize $V$ and $W$ to unit vectors $\hat{V}$, $\hat{W}$.
Now, feed $\hat{V}$ into your matrix and determine the output in terms of $\hat{V}$, $\hat{W}$ using dot products $$R\hat{V} = \{(R\hat{V})\cdot\hat{V}\}\hat{V}+\{(R\hat{V})\cdot\hat{W}\}\hat{W}=\alpha\hat{V}+\beta\hat{W},$$ and compare to the action of a right-handed rotation about $A$ through an angle $\theta$: $$\hat{V} \mapsto \cos\theta \hat{V}+\sin\theta \hat{W}.$$
• I assume you have $R$ in a matrix form; at least that's what you indicate. And you have found the eigenvector $A$ of $R$, as you stated. So you know $R$ is a rotation about $A$, but you don't know the angle. The vectors orthogonal to $A$ will stay orthogonal to $A$ under the rotation $R$. So you need a local basis in that plane, which is what $\hat{V}$, $\hat{W}$ are. Then you only need to know what $R$ does to $V$ in order to determine the angle. $R$ will map $\hat{V}$ to a linear combination of $\hat{V}$ and $\hat{W}$ where the linear factors are determined by dot products, giving the angle. – DisintegratingByParts Oct 29 '14 at 18:38