Why is it that vectors and quaternions, then, rotate by different amounts?
Consider a rotation map $\underline R(a) = q a q^{-1}$. Let's rotate this by some other map $\underline S(a) = p a p^{-1}$. The composition is
$$\underline SR(a) = pqaq^{-1} p^{-1}$$
The net quaternion of this transformation is $pq$. Now, consider the case that $p = -1$. This is a quaternion corresponding to a rotation by $\pi$. Then $pq = -q$ is distinct from the original quaternion $q$, even though the net rotation performed by $pq$ is the same as by $q$. This is the origin of saying that quaternions "double-cover" the group of rotations; they take more rotation to be rotated back to what they were, but the rotations they themselves represent still go as you'd expect.
Edit: a useful setting to understand quaternions is the geometric (or clifford) algebra.
Geometric algebra uses a geometric product of vectors. Some of the relevant properties of this product are:
- Anticommutativity of orthogonal vectors: If $u,v$ are orthogonal, then $uv = -vu$.
- Reduction to scalar product: for any vector $u$, $uu$ is a scalar, and $uu$ (or $u^2$) is equal to $|u|^2$.
- Associativity: if $a, b, c$ are vectors, then $a(bc) = (ab)c$.
Products of $k$ orthogonal vectors are called $k$-blades. Vectors are $1$-blades, and just as each vector is associated with a subspace (a line), any $2$-blade may be associated with a planar subspace, any 3-blade with a volumetric subspace, and so on. The algebra of blades allows you to talk about these objects in a more natural and easy-to-use manner than traditional linear algebra.
Why am I talking about geometric algebra, though? Because GA is a useful setting to understand quaternions.
The geometric product makes it easy to write reflection maps. Let $\underline N$ be such a reflection map, associated with reflection across a plane with normal $n$. Then the reflection of a vector $a$ is
$$\underline N(a) = -nan^{-1}$$
It's commonly known that two reflections form a rotation, so let's do it twice. Given another vector $m$, we can build a rotation like so:
$$\underline R(a) = mn a n^{-1} m^{-1}$$
The quantity $mn$ is the GA analogue of a quaternion, here called a rotor. Unlike quaternions, the number of elements of a rotor varies based on the base vector space. In 2d, for instance, rotors have only two components, and they can be identified with complex numbers.
Yes, you read correctly: GA rotors are the natural setting for both complex numbers and quaternions, as well as for describing rotations in higher dimensional spaces!
In any 2 or 3 dimensions, rotors are linear combinations of scalars and 2-blades. Each unit 2-blade squares to a negative number. Example: let $e_1, e_2$ be an orthonormal basis of $\mathbb R^2$. Then $(e_1 e_2)^2 = e_1 e_2 e_1 e_2$. Using anticommutativity, we see that $e_1 e_2 e_1 e_2 = -e_1 e_2 e_2 e_1$. Using the reduction to scalar product, this resolves to $-1$. $e_1 e_2$ directly represents the $xy$-plane, if you will.
That rotors in GA involve 2-blades (or bivectors) instead of regular vectors is a critical distinction from quaternions: it means that you no longer identify the "imaginary part" of a quaternion with a vector. While there is always a correspondence between the 2-blade part of a quaternion and a vector, they are not the same and should not be confused with each other.
With that in mind, we can write a rotor $q = s + B$, where $s$ is the scalar part and $B$ is the bivector (2-blade) part. Then, the multiplication of two quaternions follows the distributive property:
$$q_1 q_2 = (s_1 + B_1)(s_2 + B_2) = s_1 s_2 + s_1 B_2 + B_1 s_2 + B_1 B_2$$
At this point, it's common to adopt some shorthands for different terms in a geometric product: for instance, two scalars multiplying each other always produce a scalar, so you can see that $s_1 s_2$ is scalar. There is a scalar part in $B_1 B_2$ also. This is usually denoted with $\cdot$, the dot product. There is also a 2-blade term in $B_1 B_2$. We usually say that
$$B_1 B_2 = B_1 \cdot B_2 + B_1 \times B_2$$
where $B_1 \cdot B_2$ is defined as the scalar part and $B_1 \times B_2$ is defined as the bivector part. You can see this starts assembling the pieces of the formula you're concerned about; the terms arrange themselves merely as a fact of algebra.
So, if you want to understand how quaternions perform rotations, my GA perspective tells me it's good enough to understand how single vectors are used to perform reflections instead. To me, that's simple enough: the product $nan^{-1}$ can be broken down as follows: $na$ breaks $a$ into a parallel part (which becomes a scalar, from the dot product) and a perpendicular part (which forms a bivector with $n$). Multiplying by $n^{-1}$ on the wrong side maintains the parallel part while changing the sign of the perpendicular part. The overall minus is then required to get the right result for a reflection. A rotation just does this process twice.