how does the dot product determine similarity? I want to know how the dot product can determine whether two vectors are similar? I know that the formula $$\cos(\theta) = \frac{u \cdot v }{ ||u||\,||v||}$$ means something, but don't know what.
 A: The dot product of two vectors $\mathbf{u}$ and $\mathbf{v}$ is defined as
$$\mathbf{u}\cdot\mathbf{v} = |\mathbf{u}|\,|\mathbf{v}|\cos \theta$$
It's perhaps easiest to visualize its use as a similarity measure when $|\mathbf{v}|=1$, as in the diagram below, where $\cos\theta = \mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|\,|\mathbf{v}| = \mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|$.

Here you can see that when $\theta=0$ and $\cos\theta=1$, i.e. the vectors are colinear, the dot product is the product of the magnitudes of the vectors. When $\theta$ is a right angle, and $\cos\theta=0$, i.e. the vectors are orthogonal, the dot product is $0$. In general $\cos\theta$ tells you the similarity in terms of the direction of the vectors (it is $-1$ when they point in opposite directions). This holds as the number of dimensions is increased, and $\cos\theta$ has important uses as a similarity measure in multi-dimensional space.
A: Here is the geometric intuition.
Dot product between $u$, $v$ can be interpreted as projecting $u$ onto $v$ (or vice-versa), and then taking product of projected length of $u$ ($|u|$) with length of $v$ ($|v|$).
When $u$ is orthogonal to $v$, projection of $u$ onto $v$ is a 0 length vector, yielding 0 product. If you visualize all possible rotations of $u$ while keeping $v$ fixed, the dot product gives:


*

*Zero value when $u$ is orthogonal to $v$ as the projection of $u$ onto $v$ yields a vector of zero length. This corresponds to the intuition of zero similarity.

*Largest value of $|u||v|$ when $u$ and $v$ point in the same direction.

*Lowest value of $-|u||v|$ when $u$ and $v$ point in opposite direction.


Dividing the $u \cdot v$ by max value of $|u||v|$ nicely limits the range to $[-1, 1]$, making it scale invariant.
