The rotations that transform one circle onto another in an Euclidean affine space of dimension 3? In the Euclidean affine space of dimension three, we have two circles $C_1$ and $C_2$ of the same radius and non-coplanar axes, how to determine all rotations that transform $C_1$ onto $C_2$ ?
I found that there is only one rotation, I am not sure if my proof is correct:
A rotation that maps circle $C_1$ onto circle $C_2$ must map the center $O_1$ of $C_1$ onto the center $O_2$ of $C_2$. Its axis must therefore be orthogonal to the line joining the two centers and equally distant from them. And given such an axis, the rotation about it that maps $O_1$ to $O_2$ is unique. Thanks for your help.
 A: For any rotation, we need an axis of rotation, a point $p_0$ that lies on the axis of rotation, and a rotation angle $\theta$.  The axis of rotation and $\theta$ will determine the rotation matrix $R$ that together with $p_0$ determines the image of rotation of various vectors.
The matrix $R$ is given by
$R = a a^T + (I - a a^T) \cos \theta + S_a \sin \theta $
The above equation is known as the Rodriguez rotation formula.  See also here.
Where $a$ is a unit vector along the axis of rotation, and the skew-symmetrix matrix is given by
$S_a = \begin{bmatrix} 0 && - a_z && a_y \\ a_z && 0 && - ax \\ -a_y && a_x && 0 \end{bmatrix} $
From the properties of rotation, we know that if $v_2 = R v_1 $ then the axis of rotation is perpendicular to the vector $(v_2 - v_1)$.
In this problem, we have two pairs of vectors, the first pair is related by,
$O_2 = p_0 + R (O_1 - p_0)$
So that $ (O_2 - p_0) = R (O_1 - p_0) $
and also we have $n_2 = R n_1$ , where $n_1$ and $n_2$ are the unit vectors perpendicular to the planes of the circles.
Hence, the axis of rotation is perpendicular to $(O_2 - O_1)$ and also perpendicular to $n_2 - n_1$, therefore, we have to take the axis of rotation to be the unit vector along the cross product $(O_2 - O_1) \times (n_2 - n_1) $.
So, we have the axis $a$ now.  Next, we want to find the angle of rotation, and for that we can project the vectors $n_1$ and $n_2$ onto the plane $a^T v = 0 $, the angle of rotation is the angle between the projections of $n_1$ and $n_2$ onto this plane.
With this, the matrix $R$ is now fully specified.  But we still have to determine a point $p_0$ where the axis passes.  So we write
$O_2 - p_0 = R (O_1 - p_0) $
from which,
$ (I - R) p_0 = O_2 - R O_1 $
This system of three equation in three variables (the coordinated of $p_0$) is rank deficient, meaning that the matrix $(I - R)$ has only two independent rows, and its determinant is zero.  So the solution to the system will be a whole line of the form $p_0 = x_0 + t x_1 $ where $t$ is arbitrary, and $x_1$ is along the axis of rotation $a$ found above.
Any $p_0$ on this line is acceptable.
Now the rotation is fully specified, and as the author of the question mentioned, it is unique.
A: I don't agree with your reasoning. Your problem is that saying "Its axis must therefore be orthogonal to the line joining the two centers and equally distant from them."doesn't determine a unique line (rotation axis). In fact, if $P$ is the perpendicular bissector plane of the line segment joining the two centers, $\color{red}{any}$ line belonging to plane $P$ could be a candidate for being this "axis".
Here is the way I would consider the issue:
A) Existence of such a rotation.
Let us use the following notations for circles $C_1,C_2$: $O_1,O_2$ for their centers and $U_1, U_2$ resp. for the unit normal vectors of their planes.
Consider two steps:

*

*First step: operate on circle $C_1$ the translation by vector $\vec{O_1O_2}$ sending $O_1$ onto $O_2$. In this way, the image of the first circle and the second circle belong to a same sphere.


*Second step: use rotation $R$ bringing $U_1$ onto $U_2$. The rotation axis is directed by cross product $U_3:=U_1 \times U_2$ and its angle is $\theta:=\operatorname{arccos}(U_1.U_2)$. This rotation can be expressed, using unit vector $a=U_3/\|U_3\|$ by Rodrigues formula (which can be found in the other answer):
$$R = a a^T + (I - a a^T) \cos \theta + S_a \sin \theta$$
B) $\color{red}{NON-Unicity \ }$ of this solution:
The (affine space) rotation found above can be combined with a rotation that - say - transforms the second circle into itself.
Here is an example of cases where two different affine rotations
$$V \to RV+U \tag{1}$$
transforms the blue circle (center in $(0,0,0)$, axis Oz, radius 1) into the cyan circle (center in $(1,1,1)$, axis parallel to Ox, radius 1)(see figure) where
$$R=\begin{pmatrix}0&0&-1\\0&1&0\\1&0&0\end{pmatrix} \ \ \text{or} \ \ R=\begin{pmatrix}0&0&-1\\c&c&0\\c&-c&0\end{pmatrix} \ (c=\sqrt{2}/2)$$
$$\text{and} \ U=\begin{pmatrix}1\\1\\1\end{pmatrix}$$

