3D rotation matrix question: Extension of Wahba's Problem This is an extended version of Wahba's Problem.
Let $R_x(\alpha)$ denote a 3D rotation matrix around the axis $x$ by an amount $\alpha$.
For given unit vectors $\{u_k\}_{k=1}^3$ and $\{v_k\}_{k=1}^3$ I want to solve the system
$$
RR_{u_k}(\alpha_k)v_k=v_k, \quad 1\leq k\leq 3.$$
The variables that I can solve over are the three angles $\alpha_k$ as well as a full rotation matrix $R$. Clearly, $R=I$ and $\alpha_k=0, \forall k$ is one trivial solution.
Wahba's problem arises by setting $\alpha_k=0, \, \forall k$ and then asking for the best fit of $R$ but where the RHS vector $v_k$ is replaced with $u_k$. (Wahba's also have weights).
However, numerically, I find that there are other non-trivial solutions.
Using a very lengthy brute-force (analytical) method, I can establish that for the a particular choice $u_1=u_2=[1\; 0\; 0]^{\mathrm{T}}$ and $u_3=[0\; 1\;0]^{\mathrm{T}}$ there are either 2 or 4 solutions (including the trivial one) depending on the vectors $v_k$. (Other number of solutions occur for a set of $v_k$'s of measure zero).
However, I cannot crack the general case of arbitrary $u_k$'s. I feel that there could be an elegant geometric argument.
 A: EDIT: The derivation below is not entirely correct. In particular, the constructions described should be carried out on the sphere, but they do not carry over this simply to the plane, so the pictures are slightly off. Nevertheless, it shows a possible line of reasoning, that maybe can be adjusted to a correct statement.
In fact, there is a (not necessarily simple) geometric way to see this problem.
Write $v_k'=R_{u_k}(\alpha_k)v_k$ for $k=1,2,3$.
First we have the following claim:

*

*If the angle between $v_1$ and $v_2$ is the same as the angle between $v_1'$ and $v_2'$, and similarly for $v_1,v_3$ and $v_2,v_3$, then there is an orthogonal map $R$ such that $Rv_k'=v_k$ for all $k=1,2,3$.

Proof of claim: Note that these angles are equal iff the pairwise inner products are invariant. If $v_1,v_2,v_3$ span $\mathbb{R}^3$, then this directly means that the (linear extension of the) map $(v_1,v_2,v_3)\mapsto(v_1',v_2',v_3')$ is orthogonal. This proves most cases to be considered. If $v_1,v_2,v_3$ do not span the space, choose some vector $v_4$ orthogonal to the plane they span and consider the map $(v_1,v_2,v_3,v_4)\mapsto (v_1',v_2',v_3',v_4)$ (i.e. $v_4$ is invariant). This map is also orthogonal. Hence, the claim is proven.
(Note that we do not guarantee that $\det(R)=1$, I am not sure if this is always the case)
Now notice that all we are doing takes place on the unit sphere. We can stereographically project the sphere to the plane, and view the problem as a problem of 2D geometry. We have three given points $u_1,u_2,u_3$ and three arbitrary points $v_1,v_2,v_3$.
When $v_k'$ rotates around $u_k$, it can be anywhere on the circle centered around $u_k$ through $v_k$. Furthermore, for $k_2\neq k$ the angle between $v_k'$ and $v_{k_2}'$ is equal to the angle between $v_k$ and $v_{k_2}$ (in $\mathbb{R}^3$) iff the distance between $v_k$ and $v_{k_2}$ is equal to the distance between $v_k'$ and $v_{k_2}'$.
We can now try to find solutions: Let $v_1'$ be arbitrary on the circle around $u_1$ through $v_1$. Let $v_2'$ and $v_2''$ be the two points on the circle around $u_2$ through $v_2$ that are distance $|v_1v_2|$ away from $v_1'$. Now let ${v_3'}_1$ and ${v_3'}_2$  be the points of distance $|v_2v_3|$ from $v_2'$ and distance $|v_1v_3|$ from $v_1$. Similarly, let ${v_3''}_1$ and ${v_3''}_2$ be the points of distance $|v_2v_3|$ from $v_2''$ and distance $|v_1v_3|$ from $v_1$.
Now we have one unknown (the position of $v_1$ on its circle) and we have a solution every time one of the points ${v_3'}_1, {v_3'}_2, {v_3''}_1$ or ${v_3''}_2$ coincides with the circle around $u_3$ through $v_3$.
You can then use geogebra to visualise this problem. I have been able to find configurations with as much as 12 possible solutions, as well as configurations with only 2 solutions (counting multiplicities).
In the images below, the fancy curves show the possible locations of the different $v_3'$s, and the circle is around $u_3$ and through $v_3$. Each crossing of the curve with the circle is a solution.

The example above has 12 possible solutions

The example above has only 2 possible solutions.
I am not sure if there are configurations with more than 12 solutions. I did find configurations for every even number of solutions between 2 and 12.
To answer your question, this problem is highly complicated, and there is probably no easy answer to it. In particular, there is probably no elegant simple geometric solution.
Of course, the problem has at least one solution. Because all figures under consideration are circles, I think one could show somehow there is always a second solution, i.e. this system always has a non-trivial solution.
A: This is a bit too long for a comment, but not quite an answer.
Here's a way to reframe the problem. Let $w_k$ denote the unit-vector in the direction of $u_k \times v_k$, so that
$$
R_{u_k}(\alpha) v_k = \cos (\alpha) v_k + \sin(\alpha) w_k.
$$
Thus, the equation of interest becomes one of solving
$$
R(\cos (\alpha_k) v_k + \sin(\alpha_k) w_k) = v_k, \quad k = 1,2,3.
$$
To put this another way, we're looking for a solution to the matrix equation
$$
R \pmatrix{v_1 & v_2 & v_3 & w_1 & w_2 & w_3} 
\pmatrix{
a_1\\&a_2\\&&a_3\\b_1\\&b_2\\&&b_3} = \pmatrix{v_1&v_2&v_3},
$$
subject to the constraint that $R$ is rotation matrix and $a_k^2 + b_k^2 = 1$ for $k = 1,2,3$.
Let $V = [v_1 \ \ v_2 \ \ v_3]$. If the $v_i$ are ordered such that $\det(V) = 1$, then $V$ is a rotation and we can substitute $S = V^{-1}R$ to get the equivalent equation
$$
S \pmatrix{v_1 & v_2 & v_3 & w_1 & w_2 & w_3} 
\pmatrix{
a_1\\&a_2\\&&a_3\\b_1\\&b_2\\&&b_3} = I
$$
A: For clarity, I will write your first $R$ as $Q$, so that the equations are $QR_{u_i}(\alpha_i)v_i=v_i$ for $i=1,2,3$.
There are certainly cases where the only solutions are trivial. For instance, when $v_i$ aligns with the axis, i.e., if $u_i=\pm v_i$, we have $R_{u_i}(\alpha_i)v_i=v_i$ for all values of $\alpha_i$. Therefore the $i$-th equation reduces to $Qv_i=v_i$. It follows that when $u_1=v_1$ and $u_2=v_2$, the rotation matrix $Q$ must leave both $v_1$ and $v_2$ fixed. If $v_1$ and $v_2$ are not parallel to each other, $Q$ must be equal to $I$.
I don't know how to solve your problem, but one may remove the $\alpha_i$s in and your system of equations to obtain a simpler reformulation. Let us consider a single equation $QR_u(\alpha)v=v$ first. This is equivalent to
$$
R_u(\alpha)v=Q^Tv.\tag{1}
$$
Since $\langle u,R_u(\alpha)v\rangle=\langle R_u(\alpha)^Tu,v\rangle=\langle u,v\rangle$ for all values of $\alpha$, if $(1)$ is solvable, we must have
$$
\langle u,v\rangle=\langle u,Q^Tv\rangle.\tag{2}
$$
Conversely, if $(2)$ is satisfied, then $Q^Tv=\langle u,v\rangle u+\sqrt{1-\langle u,v\rangle^2}\,x$ for some unit vector $x\perp u$. As we also have $v=\langle u,v\rangle u+\sqrt{1-\langle u,v\rangle^2}\,w$ for some unit vector $w\perp u$, equation $(1)$ is now solved by taking $\alpha=\operatorname{sign}\left(\det(u,w,x)\right)\times|\arccos\left(\langle w,x\rangle\right)|$.
Since this is true for each equation $i$, your system of equations consequently reduces to
$$
\langle u_i,v_i\rangle=\langle Qu_i,v_i\rangle\quad\text{for $i=1,2,3$}.
$$
