How to find the rotation matrix (with no x rotation) between two rotation matrices? I need to find the rotation matrix (with no $x$ rotation) between two rotation matrices.
Given a starting rotation matrix $\textbf{R}_a$ and a setpoint $\textbf{R}_{SP}$. I need to find the rotation matrix $\textbf{R}_b$ that rotates $\textbf{R}_a$ so that the $x$ vector matches the setpoint matrix. I do not need the $y$ or $z$ vectors to match. In other words, I need to find $\textbf{R}_b$ such that the equation below holds true. See the example of what I need to achieve (black line = setpoint, red line = what I need to achieve).
$$\textbf{R}_a \textbf{R}_b \begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix} = \textbf{R}_{SP}\begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix}$$
Let's first assume that $\textbf{R}_b$ is a rotation matrix with $z-y'-x''$ rotation sequence, you would do $\textbf{R}_b = \textbf{R}_a ^{-1}\textbf{R}_{SP}$. In this case, $\textbf{R}_b = R_z(\psi) R_y(\theta) R_x(\phi)$ would have a roll component $R_x(\phi)$ ($x$ axis rotation).
However, in my case, $\textbf{R}_b$ cannot have any $x$ axis rotation, meaning $\textbf{R}_b = R_z(\psi) R_y(\theta)$. How can I find $\psi$ and $\theta$ so that the statement above holds true?
I know this is possible as I have tried brute forcing in Matlab (brute force example) with different rotation matrices and it seems to be possible. I just do not know how to actually work out the angles numerically.
Note that this is what I'm using:
$$ R_{z}(\psi )= \begin{bmatrix} \cos{\psi} & -sin{\psi} & 0 \\ sin{\psi} & cos{\psi} & 0 \\ 0 & 0 & 1 \end{bmatrix}$$
$$ R_{y}(\theta )= \begin{bmatrix} \cos{\theta}&0&\sin{\theta}\\0&1&0\\-sin{\theta}&0&\cos{\theta}\end{bmatrix}$$
$$ R_{x}(\phi )= \begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos{\phi} & -sin{\phi} \\ 0 & \sin{\phi} & cos{\phi} \end{bmatrix}$$
Thank you so much in advance for your help.
 A: You want to compute a rotation matrix $R = R_z(\psi) R_y(\theta)$ , which is a rotaton about $y$ followed by a rotation about $z$.  It follows that,
$R = \begin{bmatrix} \cos \psi && -\sin \psi && 0 \\
\sin \psi && \cos \psi && 0 \\ 0 && 0 && 1 \end{bmatrix}
\begin{bmatrix} \cos \theta && 0 && \sin \theta \\
0 && 1 && 0 \\ -\sin \theta && 0 && \cos \theta \end{bmatrix}$
Multiplying out, we find that
$R =\begin{bmatrix} \cos \psi \cos \theta && -\sin \psi   && \cos \psi \sin \theta \\
  \sin \psi \cos \theta   &&    \cos \psi &&    \sin \psi \sin \theta \\ - \sin \theta &&  0 && \cos \theta \end{bmatrix}$
You want to multiply $R_a$ on the left by $R_b$, so  $R = R_b R_a$ has its first column aligned with the first column of $R_{SP}$, a given rotation matrix$.
So we'll set $R_b R_a = R$
The only restriction on $R$ is that its first column is equal to the first column of $R_{SP}$ (which is a unit vector).
Suppose that the first column of $R_{SP}$ is $[x_1, x_2, x_3]^T$ , then by comparing this with the first column of $R$, it follows that
$\theta = \sin^{-1}(-x_3)$, and $\psi = \text{ATAN2}(x_1/\cos \theta, x_2/\cos \theta) $
Now matrix $R$ is fully specified.  We can now compute $R_b = R {R_a}^{-1} = R {R_a}^T$
