# Where is the square in the Least square regression method?

I'm having a serious doubt in the least square regression problem. I guess its got to do with the notation of norm. Is the least square formulation $||b - \mathbf{A}x||^2$ or is it $||b - \mathbf{A}x||_2$. Or are they the same ?

-

Usually it is $||b - \mathbf Ax||_2$, but the problem of minimizing $||b - \mathbf Ax||_2$ is the same as minimizing $||b - \mathbf Ax||_2^2$. The notation $||b - \mathbf Ax||$ is short for $||b - \mathbf Ax||_2$ when it is understood that the norm being used is the $2$-norm.
Can you explain or point me to some article which explains why the problem of minimizing $$||b - \mathbf Ax||_2$$ is the same as minimizing $$||b− \mathbf Ax||_2^2$$ . Will they always result in the same answer ? – karthik A Aug 31 '12 at 3:36
karthik, do you not see that if you have some function $f$ that takes non-negative real values then minimizing $f$ is the same as minimizing $f^2$? – Gerry Myerson Aug 31 '12 at 3:38
The function $x \mapsto x^2$ is a strictly increasing function when $x \ge 0$. That means if $x, y \ge 0$, then $x \le y$ if and only if $x^2 \le y^2$. – Tunococ Aug 31 '12 at 3:46
Suppose $v=(a,b,c)$. Then $\|v\|$ and $\|v\|_2$ are both common notations for $\sqrt{a^2+b^2+c^2}$. If you minimize $\|v\|_2$ then you also minimize $\|v\|^2$, and vice versa.