Conjugate elements of $SO(3)$ group Composition of two rotations in 3d space yields another rotation $$R_1 R_2 = R_3, $$
and I can understand this by help of some figures in my book.

So, the rotations in 3d space forms group. Then in my book, the author says that for two rotations $R$ and $R'$ with same angle but different rotation axis, they are connected by 
  $$ R' = QRQ^{-1}.$$
  Where $Q$ rotates the rotation axis of $R$ into $R'$. How do we prove this?

Graphically, I can understand this. $Q$ just changes rotation axis $R$ into $R'$, so after all steps, the rotation axis comes to initial position. Next, since $R$ and $R'$ has the same rotation angle, any rigid body is rotated same angle. So I think $R'$ and $QRQ^{-1}$ would give same result.
But How do I prove this by mathematically?
 A: To prove this sort of identity it helps to start with infinitesimal rotations:
$$
R\vec v = 1+d\phi\,\hat n\times\vec v\\
R'\vec v = 1+d\phi\,\hat n'\times\vec v\\
$$
Let $Q$ be a rotation that takes $\hat n$ to $\hat n'$:
$$
Q\hat n=\hat n'
$$
Now let's evaluate $Q^{-1}R'Q\vec v$:
$$
Q^{-1}R'Q\vec v=Q^{-1}(1+d\phi\,\hat n'\times Q\vec v)\\
=1+d\phi\,Q^{-1}(\hat n'\times Q\vec v)
$$
This last expression is equal to $R$ as long as the following is true:
$$
\hat n\times\vec v = Q^{-1}(\hat n'\times Q\vec v) = Q^{-1}(Q\hat n\times Q\vec v)
$$
This identity is the rotational invariance of the cross product and a simple way to prove it is to write it in index notation and evaluate it for a couple of cases:
$$
\epsilon_{pkq}Q_{km}Q_{pi}Q_{qj}=\epsilon_{mij}
$$


*

*If any of $m$,$i$,$j$ match then you're taking the cross product of a column of $Q$ with itself so the result is 0. This matches what happens on the right side

*If $m,i,j$ is $1,2,3$, the left side is $\det Q=1$, matching the right side

*If you permute any of the indices on the left the expression changes sign, like on the right
A: Here is hints to one possible proof:


*

*In the notation of Ref. 1, there exists rotation axis vectors $\vec{\alpha},\vec{\alpha}^{\prime}\in \mathbb{R}^3$ with $|\vec{\alpha}|=|\vec{\alpha}^{\prime}|$ such that
$$ R~=~ e^{i \vec{\alpha}\cdot \vec{L}}\quad\text{and}\quad R^{\prime}~=~ e^{i \vec{\alpha}^{\prime}\cdot \vec{L}}.$$

*Choose rotation axis vector $\vec{\beta}\in \mathbb{R}^3$ perpendicular to both $\vec{\alpha}$ and $\vec{\alpha}^{\prime}$ with length 
$$|\vec{\beta}|~=~\angle(\vec{\alpha},\vec{\alpha}^{\prime}) $$
given by the angle between $\vec{\alpha}$ and $\vec{\alpha}^{\prime}$, such that $$Q~=~ e^{i \vec{\beta}\cdot \vec{L}}\quad\text{and}\quad Q\vec{\alpha}~=~ \vec{\alpha}^{\prime}.$$

*Prove that $$QRQ^{-1}~=~\exp\left[i \vec{\alpha}\cdot Q \vec{L} Q^{-1}\right]~=~\exp\left[i \vec{\alpha}\cdot e^{i \vec{\beta}\cdot [\vec{L}, ~\cdot~]}\vec{L} \right] $$
$$~=~\exp\left[i (Q\vec{\alpha})\cdot \vec{L} \right]~=~e^{i \vec{\alpha}^{\prime}\cdot \vec{L}}~=~R^{\prime}.$$
References:


*

*G. 't Hooft, Introduction to Lie Groups in Physics, lecture notes, chapter 3. The pdf file is available here.

A: For me, the easiest way to prove this is to use matrices and eigenvalues.
Rotations around an axis are  exactly  3x3 real matrix satisfying $R^T=R^{-1}$ and $\det R=1$.
To see that, note that a rotation transform the standard basis in a new one, keeping vector ortogonal and with length=1, so $R^T=R^{-1}$, and keeps to orientation of the basis, so $det R=1$.
Conversely, if $R$ is a 3x3 real matrix such that $R^T=R^{-1}$, then they eigenvalues (real or complex) have always modulus 1  (complex eigenvalues must be conjugate pairs since $R$  is real).
Therefore, for 3x3 real matrices with $R^T=R^{-1}$ and $\det R=1$ the eigenvalues must be $a+ib$,$a-ib$, 1 with $a^2+b^2=1$ (the case a=1 is the identity, all eigenvalues are 1,  the case $a=-1$ the eigenvalues are -1,-1,1).
Then, there is a basis of eigenvectors of $R$, $v,u,w$ associated to eigenvalues  $a+ib$,$a-ib$, 1.
If $b\not=0$ (a pair of non real eigenvalues), you can take $v^*=\frac{u+v}{2}$,
$u^*=\frac{u-v}{2}$ and check that they satisfied $Rv^*=av^*-bu^*$ and $Ru^*=av^*+bu^*$, so the matrix in the basis $v^*,u^*,w$ is
$$Q=\begin{bmatrix}a&-b&0\\
b&a&0\\
0&0&1
\end{bmatrix}$$
which is a rotation of angle $\theta$, $\cos\theta=a$, around the axis $w$ (you can pick $v^*,u^*,w$ of norm=1).
Indeed, taking $P$ the matrix with columns $v^*,u^*,w$ (or $v^*,u^*,-w$ to get a basis with an correct orientation) you prove indeed that  $R=P^{-1}QP$, with $P$ again a rotation ...
