Exponentiating only the rotational component of SE(3) I'm studying the SE(3) group for an engineering application and I'm curious about the difference between $SE(3)$ and $SO(3) \times \mathbb{R}^3$.
Practically, I'd like to optimize in the tangent space of $SE(3)$ and differentiate through its $\mathrm{Exp}$ operator. I'm wondering if I can get away with just differentiating through the $\mathrm{Exp}$ of $SO(3)$ and leaving the translation component untouched.
The paper I linked has the following phrase on page 44


*

*SE(3) is diffeomorphic to SO(3) × R3 as a manifold, where each element is described by 3 · 3 + 3 = 12 coordinates (see §7.2).


*SE(3) is not isomorphic to SO(3) × R3 as a group, since the group multiplications of both groups are different. It is said that SE(3) is a semidirect product of the groups SO(3) and R3.

I'm not sure what the implication is of "diffeomorphic as a manifold" and "not isomorphic as a group".
Is $\mathrm{Exp}(SO(3)) \times \mathbb{R}^3$ still a valid retraction of $SE(3)$ ? Why?
 A: Summary of comments.
Any Lie group $G$ has a maximal compact subgroup $M$ (unique up to conjugacy), and $G$ is diffeomorphic to $M\times\mathbb{R}^r$ where $r=\dim G-\dim M$. Diffeomorphic means there is a differentiable one-to-one correspondence, so we can parametrize $G$ using $M\times\mathbb{R}^r$. However, the product group structure on $M\times\mathbb{R}^r$ does not match the group structure of $G$ in general (they are not isomorphic as groups).
In particular, the special Euclidean group $\mathbb{SE}(3)$ of rigid motions is diffeomorphic to $\mathrm{SO}(3)\times\mathbb{R}^3$, but not isomorphic. This means every rigid motion is uniquely expressible as a rotation followed by a translation (or vice-versa, as long as you're consistent).
Indeed, we can identify $\mathbb{R}^3$ with a subgroup of $\mathbb{SE}(3)$ (namely, the group of translations), and then $\mathbb{SE}(3)$ is a semidirect product $\mathbb{R}^3\rtimes\mathrm{SO}(3)$. This means the subgroup $\mathbb{R}^3$ is normal (conjugation-invariant as a set).
This is better than what you can expect for a general Lie group $G$: in general, you can't expect the $\mathbb{R}^r$ to even correspond to any subgroup (although I expect it is the bijective image of the exponential map restricted to a vector subspace of the lie algebra) or correspond to a normal subset (with the Iwasawa decomposition for $\mathrm{SL}_2\mathbb{R}$, for example, the upper triangular matrices are not conjugation-invariant).
It is also true every rotation is expressible as an exponential of a traceless antisymmetric matrix, i.e. an element of the lie algebra $\mathfrak{so}(3)$, but not uniquely. Every element of $\mathbb{R}^3$ correspond to an element of $\mathfrak{so}(3)$ via the cross product: that is, if $v\in\mathbb{R}^3$, then $T_v(u)=v\times u$ is a linear transformation expressible as a $3\times 3$ matrix which is in $\mathfrak{so}(3)$, namely
$$ \begin{bmatrix}
x \\ y \\ z
\end{bmatrix}
\quad\longleftrightarrow\quad 
\begin{bmatrix}
 0 & -z & y \\
 z & 0 & -x \\
-y & x & 0
\end{bmatrix} $$.
What helps us understand the exponential map $\exp:\mathfrak{so}(3)\to\mathrm{SO}(3)$ is the observation $\exp(T_v)$ will be a rotation around the (oriented) axis $v$ (according to the right-hand rule) by an angle of $\|v\|$. In particular, if we get $w$ from $v$ by increasing or decreasing $v$'s magnitude by $2\pi$, while maintaining its direction (or reversing so magnitude isn't negative), we find $\exp(T_u)=\exp(T_v)$; the exponential map is not injective, and the lie algebra element used to represent a rotation is not unique.
However, the lie algebra is unique if you restrict it to being $T_v$s for $v$ in the closed ball of radius $\pi$, except for antipodal points on the boundary define the same rotation under the exponential map.
Note the non-uniqueness is not related to the spin group $\mathrm{Spin}(3)$, which also goes by $S^3$ or $\mathrm{Sp}(1)$ or $\mathrm{SU}(2)$, though it is true for every element of $\mathrm{SO}(3)$ there are two distinct elements of $S^3$ corresponding to it. Even in $S^3$ itself, which is has no nontrivial cover i.e. is simply connected, elements ("versors" in the quaternion algebra) are expressible with Euler's formula and we see the vectors used to represent versors are not unique (unless, as mentioned, we restrict to the open ball of radius $\pi$, in which case only $-1$ is not represented).
