Show that if $\Omega_1,\Omega_2$ are tangent vectors of curves in $SO(3)$ then $[\Omega_1,\Omega_2]\in\mathfrak{so}(3)$ Let $so(3)$ be the Lie Algebra of $SO(3)$ and $R\in SO(3); \Omega_1,\Omega_2 \in so(3)$ and $\Omega_n = \frac{d}{dt}R_n(t)$ at the point $t=0$. So $\Omega_n$ is the tangent vector of the curve $R_n(t)$ at $t=0$.
To show that the Lie bracket $[\Omega_1,\Omega_2]$ is an element of $so(3)$ my professor wrote:
$$
\tag{1}[\Omega_1,\Omega_2] = \left.\frac{d}{dt}R_1(t)\Omega_2 R_1(t)^{-1}\right|_{t=0}
$$
I am trying to understand that, but I cannot see why this shows that the commutator is an element of $so(3)$.
I understand that for this tangential vector to be in $so(3)$, the product $R_1(t)\Omega_2 R_1(t)^{-1}$ has to be a parametrized curve in $SO(3)$.
But I do not see why it is a curve in $SO(3)$.
 A: We want to show that
$$
\tag{1}[\Omega_1,\Omega_2] = \left.\frac{d}{dt}R_1(t)\Omega_2 R_1(t)^{-1}\right|_{t=0}
$$
We can start doing this by computing:
\begin{eqnarray*}
\left.\frac{d}{dt}\left(R_1(t)\Omega_2 R_1(t)^{-1}\right)\right|_{t=0} &=&
\left.\left[\frac{d}{dt}R_1(t)(\Omega_2 R_1(t)^{-1})\right]\right|_{t=0} + \left.\left[(R_1(t)\Omega_2) \frac{d}{dt}R_1(t)^{-1}\right]\right|_{t=0} \\
&=&
\left[\Omega_1\Omega_2 R_1(0)^{-1}\right] + \left.\left[R_1(t)\Omega_2 \frac{d}{dt}R_1(t)^{-1}\right]\right|_{t=0} \\
&=&
\left[\Omega_1\Omega_2 R_1(0)^{-1}\right] + \left.\left[R_1(t)\Omega_2 (-R_1(t)^{-1})\left(\frac{d}{dt}R_1(t)\right)R_1(t)^{-1})\right]\right|_{t=0} \\
&=&
\left[\Omega_1\Omega_2 R_1(0)^{-1}\right] - \left.\left[R_1(t)\Omega_2 R_1(t)^{-1}\left(\frac{d}{dt}R_1(t)\right)R_1(t)^{-1})\right]\right|_{t=0} \\
&=&
\left[\Omega_1\Omega_2 R_1(0)^{-1}\right] - \left[R_1(0)\Omega_2 R_1(0)^{-1}\left.\left(\frac{d}{dt}R_1(t)\right|_{t=0}\right)R_1(0)^{-1})\right] \\
&=&
\left[\Omega_1\Omega_2 R_1(0)^{-1}\right] - \left[R_1(0)\Omega_2 R_1(0)^{-1}\Omega_1R_1(0)^{-1})\right] \\
\end{eqnarray*}
See Derivatives of Inverse Matrix for details on the third line's derivation.
There isn't much further we can go without knowing more details about $R_1$.  I actually think it might have been defined as
$$R_i(t) = e^{\Omega_it}$$
But it's only necessary to require $R_i(0) = I$.  Then the final line above gives $\Omega_1\Omega_2 - \Omega_2\Omega_1$.

Update to include additional requirements:

The curve is in $SO(3)$

This is not always true.  A Lie group is a subgroup of $GL$ but a Lie algebra always contains the $0$ matrix.  It maps to the identity of the Lie group.  So if $\Omega_2 = 0$ in the above, then the curve $C(t) = 0$ does not lie in $SO(3)$.

Show $[\Omega_1, \Omega_2]$ is in $\mathfrak{so}(3)$

This is true independently of the above considerations.  $\Omega_1, \Omega_2$ are elements of $\mathfrak{so}(3)$ which is closed under addition and multiplication (matrices of the form $X^T = -X$).
A: Formula (1) is a bit strange indeed, $R_1(t)\Omega_2 R_1(t)^{-1}$ is not necessarily in $SO(3)$ ($\Omega_2$ could have a null determinant for example).
One way to prove the result, is to consider that (for $t \to 0$):
$$ R_1(t) = I + t \Omega_1 + t^2 W_1 + o(t^2)$$
$$ R_2(t) = I + t \Omega_2 + t^2 W_2 + o(t^2)$$
Then:
$$ R_1(t)R_2(t) = I + t (\Omega_1 + \Omega_2) + t^2 (W_1 + W_2 + \Omega_1 \Omega_2) + o(t^2)$$
$$ R_2(t)R_1(t) = I + t (\Omega_1 + \Omega_2) + t^2 (W_1 + W_2 + \Omega_2 \Omega_1) + o(t^2)$$
The you can consider $L(t) = R_1(t)^{-1}R_2(t)^{-1}R_1(t)R_2(t)$ which is a curve in $SO(3)$. Since $R_2(t)R_1(t)L(t) = R_1(t)R_2(t)$, you can develop term by term each side and find that:
$$ L(t) = I + t^2([\Omega_1, \Omega_2]) + o(t^2)$$
If you re-parametrise L with $s = t^2$ you have $L(s) = I + s([\Omega_1, \Omega_2]) + o(s)$ Thus you have found a curve in $SO(3)$ whose tangent is $[\Omega_1, \Omega_2]$.
A: Found a solution, but it has nothing to do with $(1)$:
$$
\begin{align}
[\Omega_1,\Omega_2] & = \Omega_1\Omega_2-\Omega_2\Omega_1 = (-\Omega_1^T)(-\Omega_2^T) - (-\Omega_2^T)(-\Omega_1^T) \\ & = \Omega_1^T\Omega_2^T-\Omega_2^T\Omega_1^T=(\Omega_2\Omega_1)^T-(\Omega_1\Omega_2)^T = (\Omega_2\Omega_1-\Omega_1\Omega_2)^T \\ & = -[\Omega_2,\Omega_1]
\end{align}
$$
A: Here is a possibility as to how the professor was arriving at the result that $[\Omega_1, \Omega_2]$ is an element of the Lie algebra.  Suppose that we take the above formula as the definition of the Lie Bracket:
$$
\tag{1}[\Omega_1,\Omega_2] = \left.\frac{d}{dt}R_1(t)\Omega_2 R_1(t)^{-1}\right|_{t=0}
$$
There is a proposition that for any Lie group element $A$ and Lie algebra element $X$ (of its associated Lie algebra), that $AXA^{-1}$ is also in the Lie algebra.  (Proof from Hall: $e^{t(AXA^{-1})} = Ae^{tX}A^{-1}$ and the latter is in the Lie group).
This means in the above, the curve $$C(t) = R_1(t)\Omega_2 R_1(t)^{-1}$$
lies entirely in the Lie algebra.  Hence its derivative at $t = 0$ is also in the Lie algebra (because a Lie algebra is a vector space).  So the misconception from the original question was that $C(t)$ lives in $\mathfrak{so}(3)$ instead of $SO(3)$.
