How to find 2D rotation matrix that rotates vector $\mathbf{a}$ to $\mathbf{b}$ I have two 2D unit vectors a and b. I'd like to find the rotation matrix that rotates a to b. The formulas I see online are for a rotation matrix are 
$$ 
    \left(
    \begin{matrix}
    \cos \theta & - \sin \theta \\
    \sin \theta &  \cos \theta \\
    \end{matrix}
    \right)
$$
And I can get the angle between a and b with
$$
\theta = \cos^{-1} (\mathbf{a} \cdot \mathbf{b})
$$
My problem is that that doesn't give me the direction. For example, if $\theta$ is $ \pi/2 $ when maybe the matrix should use $ -\pi/2 $
 A: You don’t need to compute the angle explicitly, or indeed refer to an angle at all.
Observe that the result of rotating any vector $(x,y)^T$ 90 degrees counterclockwise is $(-y,x)^T$. Then, using the fact that the columns of a transformation matrix are the images of the basis vectors, the matrix $$\begin{bmatrix}x&-y\\y&x\end{bmatrix}$$ represents a rotation that maps $(1,0)^T$ onto the unit vector $(x,y)^T$. Therefore, a rotation that takes the unit vector $\mathbf a=(x_a,y_a)^T$ to the unit vector $\mathbf b=(x_b,y_b)^T$ has the matrix $$\begin{bmatrix}x_b&-y_b\\y_b&x_b\end{bmatrix} \begin{bmatrix}x_a&-y_a\\y_a&x_a\end{bmatrix}^{-1} = \begin{bmatrix}x_b&-y_b\\y_b&x_b\end{bmatrix} \begin{bmatrix}x_a&y_a\\-y_a&x_a\end{bmatrix} = \begin{bmatrix}x_ax_b+y_ay_b&x_by_a-x_ay_b \\ x_ay_b-x_by_a & x_ax_b+y_ay_b\end{bmatrix}.$$
A: My solution is to regard both vectors as complex numbers. Thus $\,{\bf a}=a_x+a_yi\,$ and $\,{\bf b}=b_x+b_yi.\,$ If
both are unit vectors, then to rotate $\,\bf{a}\,$ to
$\,\bf{b}\,$ you need to multiply by
$${\bf a^{-1}b=\bar ab}=(a_x\!-\!a_yi)(b_x\!+\!b_yi)
=(a_xb_x+a_yb_y)+(a_xb_y-a_yb_x)i.$$
Since this is  a unit vector it is of the form
$\,\cos(\theta)+\sin(\theta)\,i\,$ and you can
get the rotation matrix
$$  \begin{pmatrix}
    \cos \theta & - \sin \theta \\
    \sin \theta &  \cos \theta \\
    \end{pmatrix} $$
directly from the complex number form. If you 
also need to find the angle $\theta\,$ it
is best to use the atan2 function.
A: The direction is given by a choice of basis. In the standard choice of basis, the sign of $\theta$ is that of 
$$\left|\matrix{a_x &a_y\\b_x&b_y}\right|=a_xb_y-a_yb_x$$
(If you want to rotate $b$ to $a$, the determinant should read $\left|\matrix{b_x &b_y\\a_x&a_y}\right|$, and equal to minus of the above. Phew, the maths agrees with intuition.)
A: Use the cross product instead. The $\sin$ function gives the direction as required.
$$\theta=\sin^{-1}\biggr(\frac {|\vec a\times\vec b|}{|\vec a||\vec b|}\biggr)$$
A: You can solve $$ {\bf R} = \min_{\bf R}\|{\bf Ra-b}\|_2^2 + \|({\bf 11}^T - {\bf I})\circ({\bf R}^T-{\bf R})\|_2^2$$
For matrix $\bf R$.
Where $\circ$ is a Hadamard-Schur product. This can be solved using Kronecker products.

The obvious advantage to this approach is that we can expand to a whole set of vectors:
$$ {\bf R} = \min_{\bf R}\|{\bf RA-B}\|_2^2 + \|({\bf 11}^T - {\bf I})\circ({\bf R}^T-{\bf R})\|_2^2$$
For higher dimensions this is important because for any other two vectors $${\bf Ra}_k = {\bf b}_k$$ than in the plane of rotation, the transformation should be such that ${\bf a}_k ={\bf b}_k$ (identity).
With increasing dimension of our vector space on which $\bf R$ works, the number of possible linear transformations which map $\bf a$ to $\bf b$ will be growing very fast. In fact it is only for 2D it shall be unique.
