4
$\begingroup$

I've been studying descriptive statistics and am having a hard time understanding the actual intuition behind standard deviation. I'm trying to get a practical feeling for it and so I'm trying to draw conclusions from it using a distribution of 20 numbers, from 1 to 20. I know the mean is 10.5 and the absolute average deviation is 5, which is pretty intuitive.

Now when taking the standard deviation I get the value 5.77 which still makes some sense if I think about it as the average euclidean deviation from the mean. So I imagine adding orthogonal distances and then averaging them $\frac{\sum(x_i-\bar x)^2}{n}$ and taking the square root of that at the end to get the actual average distance. The formula makes sense from an euclidean perspective. So all that being said, my questions:

1) Why would an euclidean average distance be more accurate than an absolute deviation from the mean? I actually think absolute average deviation is more accurate since it doesn't infer any direction of the values. When taking the euclidean distance, I'm pretty much saying every value is placed at a 90° angle from each other. That does not sound right. So why the Euclidean distance? (I'm aware of this article but If someone could actually explain what efficiency is that would be very helpful: https://www.leeds.ac.uk/educol/documents/00003759.htm)

2) If The advantage of using SD is because of all the math we have developed around normal distribution shapes (68%, 95%, 99,7%...) wouldn't it be better to just rewrite that model with the new average deviation?

3) Ill probably post another question in the future about this, but when calculating standard error, this standard deviation seems to get even worse, since we need corrections for finite populations. Does this make any sense?

$\endgroup$
  • 1
    $\begingroup$ Worth reading math.stackexchange.com/questions/717339/why-is-variance-squared and stats.stackexchange.com/questions/118/… - it is often the variance which has the nice properties; its square-root (the standard deviation) then benefits from being in the same units as the original data and so being a scale parameter $\endgroup$ – Henry Aug 21 '17 at 0:12
  • $\begingroup$ I think the regrettably now-deleted post by @ThomasAndrews makes interesting points, as does the link in Henry's Comment. Also, generally speaking, when you ask for 'intuition' you have to realize that what falls flat with one person may be an awe-inspiring revelation for another. Intuition is opinion. So it may not be productive to argue about the validity of intuition. $\endgroup$ – BruceET Aug 21 '17 at 0:25
2
$\begingroup$

Given a sample $X_1, X_2, \dots, X_n,$ suppose we make a 'stripchart' (also called 'dotplot') of the data. Shown below are stripcharts of three samples of size $n=5:$ $$X = (0, 2, 2, 2, 4),\, Y = (0, 1, 2, 3, 4),\, Z = (0, 0, 2, 4, 4).$$

enter image description here

The sample mean $\bar X = \frac{1}{n}\sum_{i=1}^n X_i$ can be considered as the 'balance point' or 'center of gravity' of the stripchart of a sample $X_i$ (where all dots have the same weight). The sample mean $\bar X$ is also a good estimate of the mean $\mu_X$ of the population from which the sample is randomly chosen. In particular, $E(\bar X) = \mu_X.$ In the figure, $\bar X = \bar Y = \bar Z = 2.$

One way to measure the dispersion or variability of a sample is its range $R = \max(X_i) - \min(X_i).$ For our three small datasets $R_X = R_Y = R_Z = 4,$ so the sample range is not an effective measure of dispersion for distinguishing among our three datasets.

Yet, it seems intuitively clear that the $Z_i$s are the most disperse and the $X_i$s are the least disperse. As a physical model think of a vertical shaft or axis at 2, around which a stripchart is to be spun. The moments-of-inertia increase as we move from the $X_i$ to the $Y_i$ to the $Z_i.$ Roughly speaking, that is to say, the $Z_i$ make the best flywheel.

A measure of dispersion that closely parallels the definition of moment-of-inertia is the sample variance. It is roughly the average of the squared deviations about the mean: $S_X^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2.$ For our three examples, $S_X^2 = 2.0,\, S_Y^2 = 2.5,\,$ and $S_Z^2 = 4.$ So the variances do increase appropriately as we move from the $X_i$ to the $Y_i$ to the $Z_i.$ [The positive square root $S_X$ of the sample variance $S_X^2$ is called the 'sample standard deviation', which helps to explain the notation.]

One reason for using $n-1$ instead of $n$ in the denominator of the sample variance is to make $S_X^2$ a good estimator of the population variance $\sigma_X^2.$ In particular, $E(S_X^2) = \sigma_X^2.$ [Roughly speaking, another reason is that $X$ can be viewed as a vector in 5-dimensional space; one dimension is "used" to estimate $\mu$ by $\bar X,$ leaving $n-1 = 4$ dimensions to estimate $\sigma^2.$ To make this idea precise requires a side trip into linear algebra that I won't take here.]

Note: A variety of other potential measures of sample dispersion, including the one your mention, have been advocated by respected practitioners. Several of them use absolute deviations from the center (sometimes called 'discrepancies'). Examples are $\frac{1}{n}\sum_{i=1}^n |X_i - \bar X|,\; \frac{1}{n}\sum_{i=1}^n |X_i - H_X|,$ and the sample median of $|X_i - H_X|,$ where $H_X$ is the sample median. At various times and places both of the first two have been called 'MAD' (for Mean Absolute Deviation).

While each of these may have advantages in specific applications, none of them is in widespread regular use. Objections include the difficulty of doing proofs (absolute values can lead to the need to consider cases) and the lack of easily accessible distribution theory (except perhaps via simulation). [By contrast, for normal data, $\frac{(n-1)S_X^2}{\sigma_X^2} \sim \mathsf{Chisq(\nu = n-1)},$ a widely tabled and programmed family of distributions.]

Also, population variances have the advantage that $Var(X + Y) = Var(X) + Var(Y),$ provided that $X$ and $Y$ are uncorrelated.

$\endgroup$
  • $\begingroup$ Thanks for taking a shot at answering this question, and I agree with everything you said. But when you say "A measure of dispersion that closely parallels the definition of moment-of-inertia is the sample variance.", I also think that 'MAD' also does that (as you clearly stated at the end) and so does $\frac{1}{n}\sum{(x_i-\mu)^k}$ where k is any natural number and $\sigma = \sqrt[k]{\sigma^k}$. It's still confusing to me why choose one of another. Simplicity for further calculations just won't cut it for me $\endgroup$ – Danilo Souza Morães Aug 22 '17 at 7:31
  • $\begingroup$ Adding to my last comment: When k=1 we have MAD. When k=2 we have variance. Why not k=3 then? $\endgroup$ – Danilo Souza Morães Aug 22 '17 at 7:38
  • $\begingroup$ Only $k=2$ is related to physics def'n of moment of inertia; $k = 3$ measures skewness, not dispersion. $\endgroup$ – BruceET Aug 22 '17 at 16:31
  • $\begingroup$ I had not thought of that. Ill give it some thought $\endgroup$ – Danilo Souza Morães Aug 23 '17 at 21:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.