Mechanism Behind Dot Product and Least Square

Sorry for my ignorance, but I want to know how the mechanism of finding the least square solutions or the closest points in Euclidean space works.

For example:

Find the closest point or points to $b =(−1,2)^T$ that lie on the line $x + y = 0$.

$$\frac{\left( \begin{matrix} -1 \\ 2 \end{matrix} \right) \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right)}{\left( \begin{matrix} 1 \\ -1 \end{matrix} \right) \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right)} \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right) = \left( \begin{matrix} -\frac{3}{2} \\ \frac{3}{2} \end{matrix} \right)$$

But what does the dot product between $\left( \begin{matrix} -1 \\ 2\end{matrix} \right) \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right)$ tell you? If I commute it the answer is $-3$, but what exactly is $-3$? Also, the denominator is $2$, again, what exactly is $2$ telling us here?

-

The projection onto a vector $x$ is give by $$Pv=\frac{xx^T}{||x||^2}v,$$ where $P \equiv \frac{xx^T}{||x||^2}$ is the projection operator, and $||x||^2=(x,x)$ (or $x*x$ in your notation) is the normalization factor.

-
I haven't learned projections yet, can you clarify what it exactly means? – diimension Oct 16 '12 at 2:26
Projections is exactly what I was looking for! My book never mentioned it when discussing this. Thank you very much, Chaohuang! – diimension Oct 16 '12 at 3:00

Clearly your first equation must be incorrect -- the dot product returns a scalar, so there is no way to obtain a vector on the right hand side, unless you are overloading your nomenclature.

The dot product returns the sum of the element-wise products of vectors. When you take the dot product of a vector with itself, then that dot product returns the sum of the squares of the component of that vector.

Thus, when you perform a transform such as $\frac{y^Tx}{x^Tx} = \frac{y\cdot x}{x\cdot x}$ you are dividing by the sum of the squares of $x$. The sum of the squares of a vector is, of course, the square of the distance of that vector from the origin.

-
I already know to commute the dot product. What I want to know exactly is what does that scalar, when commuting the dot product, tell you in terms of the graph? If I draw a vector of slope y = -x it will cross the origin and if I take the dot product of that slope it will equal to the scalar 2. But what is 2 exactly in terms of the graph? – diimension Oct 16 '12 at 2:22