# Find an angle to rotate a vector around a ray so that the vector gets as close as possible to another vector.

I have two Vectors: the Vector I need to move (Vector A) and the Vector I need to get as close as possible to (Vector B). I also have a Ray R (an origin + axis/direction) which Vector A should be rotated around. How do I find an Angle, that will bring Vector A as close as possible to Vector B, when rotating around the Ray R?

CONTEXT

I am building an editor extension for Unity, a custom gizmo for various transformations of objects: translations/rotations/scales. Among other things it should allow for "vertex-snap rotations" In the picture above, the tool should be able to rotate the cube in such a way around the custom axis, that Vector A and Vector B get aligned. This case is an easy one, because the axis is the Wold Y axis, I need a general solution for any axis and any kind of vectors.

The reason why I might have sounded confusing is because my tool allows to set next to any constrains on what rotations should be performed: the user can turn off X-axis rotation (to only rotate in YZ-plane), they can turn off two axes and even do the same for the object's local space. I've got a method that rotates the cube around an axis with an angle, all I need is to find the closest angle BUT for a specific axis.

• When you say that you have a ray $R$, what does it mean to rotate $A$ around $R$? Suppose that $A = (1,0,0)$ is a vector pointing along the $x$-axis, and $R$ is a ray starting at $(0,1,0)$ and going in direction $(1,0,0)$. What's the result of rotating $A$ around $R$ by 180 degrees? Now suppose that $B$ is the vector $(0,1,0)$. What's the result of rotating $B$ around $R$ by $180$ degrees? – John Hughes Dec 3 '17 at 13:26
• I'm not bright at math, I am not exactly sure whether "rotate around axis" is the same as "rotate around axis that is located at some point in space". I think just "rotate around axis" should be enough for me. – cubrman Dec 3 '17 at 16:14
• Let me be clear: I'd rather not spend time writing an answer to a question that MIGHT be the one you're asking. You say that vector A should be rotated around a ray R; before I can tell you the best way to do that, you have to tell me what that sentence means. Or perhaps you can just give the context in which this problem arises, so that I can help you figure out what it means. But this is a math site, not a mindreading one; help us out here. – John Hughes Dec 3 '17 at 16:25
• @JohnHuges, sorry for being confusing - I've updated the question with the context. – cubrman Dec 3 '17 at 17:26

$$\newcommand{\va}{{\mathbf a}} \newcommand{\vv}{{\mathbf v}} \newcommand{\vb}{{\mathbf b}} \newcommand{\vu}{{\mathbf u}} \newcommand{\vf}{{\mathbf f}} \newcommand{\vc}{{\mathbf c}}\newcommand{\ve}{{\mathbf e}}$$

Let's suppose that $\va$ and $\vb$ are the vectors you care about, and that each is a unit vector, i.e. $\| \va \| = \| \vb \| = 1$. Let's suppose that $\vu$ is similarly a unit vector pointing along the axis of your ray.

I'm going to assume that $\va$ and $\vu$ are not parallel, for if they are, no amount of rotation about $\vu$ will change $\va$, so there's no "best angle."

OK then. Let $$\vc = \va - (\va \cdot \vu) \vu \\ \ve = \frac{1}{\| \vc \|} \vc.$$ Then $\ve$ is a unit vector perpendicular to $\vu$ in the $\vu\va$-plane.

Let $$\vf = \vu \times \ve$$ as well. Then let $$\theta = \text{atan2} (\vb \cdot \vf, \vb \cdot \ve)$$ Then you want to rotate $\va$ about $\vu$ by angle $\theta$, where the sign depends on how you describe rotations about an axis (there are two reasonable choices, and they require the two different choices of $\pm$).

If you want the actual matrix of the rotation (assuming you're using column vectors, and transformations that look like $\vv \mapsto M\vv$), then let $$A = \pmatrix{ \ve & \vf & \vu}$$ where that means that the columns of $A$ are the vectors $\ve,\vf,$ and $\vu$, and let $$B = \pmatrix { c & -s & 0 \\ s & c & 0 \\ 0 & 0 & 1},$$ where $c = \cos(\theta), s = \sin(\theta)$. Finally, let $$M = A B A^t.$$

• wow, this actually worked right off the gate. I'll run a few more tests and mark as answer. – cubrman Dec 4 '17 at 14:19
• Jesus, this actually INSTANTLY works for all the situations, like immediately. – cubrman Dec 4 '17 at 14:23
• You mis-spelled my name. It's "John". :) If it didn't work, I'd be kinda embarrassed (see amazon.com/Computer-Graphics-Principles-Practice-3rd/dp/…). – John Hughes Dec 4 '17 at 21:29
• Still, it's the first time in my humble experience the thing worked right of the bat with no tweaking. Especially with angles man, that's insane! – cubrman Dec 5 '17 at 7:53
• hey would you like to check out this question: math.stackexchange.com/questions/2551894/… it continues this series. – cubrman Dec 5 '17 at 8:11

You could consider the two planes:

1. $\pi_A$ containing $A+RayR$

2. $\pi_B$ containing $B+RayR$

and then determine the angle between them.

I'm assuming that the distance between $A$ and $B$ is at a minimum when $\pi_1 \equiv \pi_2$.

Here is a sketch:

• Could you elaborate on your answer? As far as I understood, I need to build two planes, using Ray R as a normal and each vector as a point and then find the angle between planes, correct? – cubrman Dec 3 '17 at 17:50
• Don't think this will work - in my example, for instance, both planes align. – cubrman Dec 3 '17 at 17:58
• I've edited and better explained my answer. I think you should find the 2 planes that contain RayR+A and RayR+B. To find the plane $\pi_1$ for example you could calculate the normal vector "n_1" by the cross product between A and a vector $C \in \pi_1$. – gimusi Dec 3 '17 at 17:58
• when the plane coincide I think that the distance is at a minimum for rotation aroun RayR – gimusi Dec 3 '17 at 18:00
• Ok, I will check it and get back to you. – cubrman Dec 3 '17 at 18:17