Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Sorry for my ignorance, but I want to know how the mechanism of finding the least square solutions or the closest points in Euclidean space works.

For example:

Find the closest point or points to $b =(−1,2)^T$ that lie on the line $x + y = 0$.

I know the answer is

$$\frac{\left( \begin{matrix} -1 \\ 2 \end{matrix} \right) \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right)}{\left( \begin{matrix} 1 \\ -1 \end{matrix} \right) \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right)} \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right) = \left( \begin{matrix} -\frac{3}{2} \\ \frac{3}{2} \end{matrix} \right)$$

But what does the dot product between $\left( \begin{matrix} -1 \\ 2\end{matrix} \right) \cdot \left( \begin{matrix} 1 \\ -1 \end{matrix} \right)$ tell you? If I commute it the answer is $-3$, but what exactly is $-3$? Also, the denominator is $2$, again, what exactly is $2$ telling us here?

share|cite|improve this question
up vote 1 down vote accepted

The projection onto a vector $x$ is give by $$Pv=\frac{xx^T}{||x||^2}v,$$ where $P \equiv \frac{xx^T}{||x||^2}$ is the projection operator, and $||x||^2=(x,x)$ (or $x*x$ in your notation) is the normalization factor.

share|cite|improve this answer
I haven't learned projections yet, can you clarify what it exactly means? – diimension Oct 16 '12 at 2:26
Projections is exactly what I was looking for! My book never mentioned it when discussing this. Thank you very much, Chaohuang! – diimension Oct 16 '12 at 3:00

Clearly your first equation must be incorrect -- the dot product returns a scalar, so there is no way to obtain a vector on the right hand side, unless you are overloading your nomenclature.

The dot product returns the sum of the element-wise products of vectors. When you take the dot product of a vector with itself, then that dot product returns the sum of the squares of the component of that vector.

Thus, when you perform a transform such as $\frac{y^Tx}{x^Tx} = \frac{y\cdot x}{x\cdot x}$ you are dividing by the sum of the squares of $x$. The sum of the squares of a vector is, of course, the square of the distance of that vector from the origin.

share|cite|improve this answer
I already know to commute the dot product. What I want to know exactly is what does that scalar, when commuting the dot product, tell you in terms of the graph? If I draw a vector of slope y = -x it will cross the origin and if I take the dot product of that slope it will equal to the scalar 2. But what is 2 exactly in terms of the graph? – diimension Oct 16 '12 at 2:22

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.