Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Possible Duplicate:
Motivation behind standard deviation?

In statistics very often you see something of the sort: $$ \textrm{quantity}=\sqrt{\frac {\sum(x-\mu)^2} {N}} $$ to measure things like standard deviation ($\mu$ is the mean here).

It seems that just making an absolute value of the difference will give us a pretty good measure of the same thing: $$ \textrm{quantity}=\frac {\sum{\Bigl|x-\mu\Bigr|}} {N} $$

How did we end up with those squares?

share|improve this question

marked as duplicate by Rahul, Henry T. Horton, Ittay Weiss, userNaN, Zhen Lin Dec 29 '12 at 12:03

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

    
That in fact is also a measure which is used. Least squares however has the advantage it is easy to work with –  user50407 Dec 29 '12 at 1:21
    
I thought it had something to do with Euclidean norms. At least this one question on variance during a statistics course I took convinced me so. –  Joe Z. Dec 29 '12 at 2:03
    
A statistics teacher once told me that squares are good because they penalize very far values for being far; i.e. squares are a compromise between the "very forgiving" absolute values, that treat many small deviations the same as one large deviation and the "very unforgiving" maximum, that completely ignores everything but the largest deviation. I don't know if this makes any mathematical sense, or is just a retroactive justification. –  Alfonso Fernandez Dec 29 '12 at 2:27
1  
Be sure to check out stats.stackexchange.com/questions/118/… –  anon Dec 29 '12 at 3:45

4 Answers 4

When you sample $n$ values of a random variable $X$, you have $n$-values $x_1,x_2,\ldots,x_n$. You've just sampled one vector in $\Bbb{R}^n$: $\vec{x}=\langle x_1,x_2,\ldots,x_n\rangle$. The standard formula for standard deviation applies the Euclidean distance metric to this vector and its mid-vector: $\langle\overline{x},\overline{x},\ldots,\overline{x}\rangle$, then dividing by $n$ as a way to account for largness due simply to large dimension.

What is your favorite distance metric in $\mathbb{R}^n$? If it is the usual Euclidean metric, then the standard formula for standard deviation arises. If it is the taxi cab metric, then you could employ the formula that you suggest. But one answer to the question of "why the squares" is that the Euclidean distance metric is generally the natural one.

share|improve this answer

The mean $\mu$ of a set of values $\{x_1,...x_n\}$ is the unique value of $y$ which minimizes $$\sum_{i=1}^n (x_i-y)^2$$ (or equivalently, $$\frac{\sum_{i=1}^n (x_i-y)^2}{N}.)$$

The mean is the value which minimizes the total squared deviation from it, and the standard deviation is the square root of that deviation.

So the squares are built into the definition of the mean. You could redefine the mean to minimize $$ \sum_{i=1}^n |x_i-y|$$ but one immediate issue is that there is often not a unique solution (for example, with the values $\{1,3,4,5\}$, all $y$-values with $3\le y \le 4$ minimize the sum).

share|improve this answer

If you generalize the squares to some dimension $n$ it results in a cute algebraic formula $A^TAx=A^Tb$ the same unfortunately isn't the case for the absolute values. If you are a computer, it isn't such a big difference. However, if you are a human used to doing some calculus and then solving pure algebraic equations, then the sharp corner at $0$ of $|x|$ maybe irritates you enough to insist on the smoothness of $x^2$. Historically both the squares and the absolute values were used by Laplace and Gauss, but somehow the absolute values got the sharp end of the stick for a bit.

share|improve this answer

Simply because it is easier to work with analytically.

(One might ask similar questions about why we use least squares instead of least fourth powers.)

Note that when the foundations of statistics and probability were laid, there were not computers (in the modern sense). However, some people are now using absolute value approaches in lieu of squares/square roots because we are finally able to do so with modern computing power.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.