Rotating one 3d-vector to another I have written an algorithm for solving the following problem: Given two 3d-vectors, say: $a,b$, find rotation of $a$ so that its orientation matches $b$.
However, I am not sure if the following algorithm works in all cases:
1) Find axis and angle using cross product and dot product:
$$\mathbf{x}=\frac{a\times b}{||a\times b||}\\ \theta=\cos^{-1}(\frac{a\cdot b}{||a||\cdot ||b||})$$
3) Find rotation matrix using exponential map:
$$\mathbf{R}=e^{\mathbf{A}\theta} =\mathbf{I}+\sin(\theta)\cdot \mathbf{A}+\left(1-\cos(\theta)\right) \cdot \mathbf{A}^{2}$$
where $\mathbf{A}$ is a skew-symmetric matrix corresponding to $\mathbf{x}$:
$$\mathbf{A}=[\mathbf{x}]_{\times}=\begin{bmatrix}0 & -\mathbf{x}_{3} & \mathbf{x}_{2} \\ \mathbf{x}_{3} & 0 & -\mathbf{x}_{1} \\ -\mathbf{x}_{2} & \mathbf{x}_{1} & 0\end{bmatrix}$$
Notes:
The axis is computed using cross product as this gives vector perpendicular to both $a$ and $b$. Only direction of the axis is important, hence it is divided by its magnitude. However, I am not sure if $\mathbf{x}$ will always have the proper direction (the result can be $-\mathbf{x}$ instead of $\mathbf{x}$?).
The rotation matrix is computed using Rodrigues' rotation formula.
Finally, the vector $\mathbf{R}a$ should have same direction as $b$.
I have tested this numerically and it seems working, but I would like to be sure the formulas work for any two $a,b$.
 A: Let $F_n(x) = x - {2 \over n . n} n (n \ . \ x)$ be the transformation that reflects $x$ through the plane that is perpendicular to $n$. Composing two reflections gives a rotation: if the angle from $a$ to $b$ is $\phi$ then $F_b(F_a(x))$ rotates $x$ on the plane spanned by $a$ and $b$ by $2 \phi$. 
Given the normalized vectors $\hat a=a/|a|$, $\hat b=b/|b|$, $\hat c=c/|c|$, where $c = \hat a + \hat b$, the angle from $\hat a$ to $\hat c$ is $\phi$, half the angle from $\hat a$ to $\hat b$.
In fact, $F_{\hat c}(F_{\hat a}(x))$ rotates $x$ by $\phi$ on the plane spanned by $a$ and $b$. This is valid even if $a$ and $b$ are parallel. This avoids computing a cross product, inverse cosine, sine and cosine, or division by a magnitude that can be arbitrarily close to zero.
A: I'm not clear on why you have a factor of $A^2$ in your expression for $R$.  In particular, wikipedia lists the matrix form for the Rodrigues formula as
$$R = I \cos \theta  + A \sin \theta + (1-\cos \theta) x x^T$$
A: I know this is a long-dead and well-answered question, but I woke this morning thinking "All of the answers involve an "if" statement, i.e., all are discontinuous functions of the inputs. Is there a continuous answer, i.e., a continuous function 
$$
R: S^2 \times S^2 \to SO(3)
$$
that takes a pair of vectors $u, v$ to a rotation matrix $R(u, v)$ with the property that $R(u, v) u = v$?"
The paper that Tomas Moller and I wrote back in 1999, for instance, 
uses "the coordinate vector corresponding to the smallest entry of $w$" for some vector $w$, which doesn't vary continuously as a function of $w$. And I wondered, "Did we really do as well as possible, or might there have been a continuous solution?" 
The answer is "no." But the proof uses a bit of topology. 
Fix the vector $u$ (set it to be $e_1$, for instance), and look at the map
$$
K : S^2 \to SO(3) : v \mapsto R(u, v).
$$
Then compose this with the map 
$$
H : SO(3) \to S^2 : M \mapsto Mu.
$$
The composite map 
$$
H\circ K: S^2 \to S^2
$$
is $v \mapsto R(u, v)u = v$, i.e., the identity map on $S^2$. But that means that 
$$
(H\circ K)_{*} : H_2(S^2) \to H_2(S^2),
$$
the induced map on second homology, must be the identify from $\mathbb Z $ to $\mathbb Z$. But since 
$$
(H\circ K)_{*} = H_{*} \circ K_{*}$$
this map must factor through $H_2(SO(3)) = 0$, which is impossible. 
Thus: There's no continuous solution to the "rotate one vector to another" problem, a fact that I should have mentioned back in our original paper. Sigh. Hindshight is 20-20. 
A: This is the right general approach, but the corner case $\|a\times b\| \approx 0$ must be handled.
If $\theta < \epsilon,$ $R=I$.
If $\pi-\theta < \epsilon$, you can choose for $\mathbf{x}$ any vector orthogonal to $\mathbf{a}$, for instance $\mathbf{x} = \frac{\mathbf{a} \times e_i}{\|\mathbf{a}\times e_i\|}$, where $i$ is the index of the component of $\mathbf{a}$ with least magnitude.
A: I have a simpler method comes from Erigen's "Mechanics of Continua". Here R is rotational matrix that rotate vector "a" align with vector "b"
Matlab Code:
%%%%%% Rotate vector a align with vector b%%%%%%%%%%
syms ax ay az bx by bz k real

a=[ax ay az]'

au=a./sqrt(ax^2+ay^2+az^2)

b=[bx by bz]'

bu=b./sqrt(bx^2+by^2+bz^2)

R=[bu(1)*au(1) bu(1)*au(2) bu(1)*au(3);

    bu(2)*au(1) bu(2)*au(2) bu(2)*au(3)

    bu(3)*au(1) bu(3)*au(2) bu(3)*au(3)]

% You can verify it by type

c=R*a

cu=c./sqrt(c(1)^2+c(2)^2+c(3)^2)

simple(bu-cu)

%the result is zero means c(a after rotation) and b are aligned with each other.  

simple(sqrt(c(1)^2+c(2)^2+c(3)^2)-sqrt(c(1)^2+c(2)^2+c(3)^2))

%the result is zero means c(a after rotation) and a are of the same length


%%%%%%%End%%%%%%%%%%%%

A: function [ R ] = RotAtoB( a,b )
    x = [a(2)*b(3) - b(2)*a(3);a(3)*b(1) - b(3)*a(1);a(1)*b(2) - b(1)*a(2)];
    x = x/norm(x);
    theta = acos(a'*b/(norm(a)*norm(b)));
    A = [0    -x(3)  x(2)
         x(3)   0   -x(1)
        -x(2)  x(1)   0  ];
    R = eye(3) + sin(theta)*A + (1-cos(theta))*A^2;
end

Calculating the powers of matrix :
Nth power of a square matrix and the Binet Formula for Fibonacci sequence
A: For a solution that works in any dimensions and leaves vectors perpendicular to $a$ and $b$ unchanged, see Does this orthogonal matrix parametrization have a name?
