Say $X$ is a random variable with a binomial distribution where $n = 3$ and $p = 0.5$, eg. the number of heads we get when we flip a coin three times.

So this distribution has the values -

  • $0$ with probability $\frac{1}{8}$
  • $1$ with probability $\frac{3}{8}$
  • $2$ with probability $\frac{3}{8}$
  • $3$ with probability $\frac{1}{8}$

The mean is $1.5$.

Now if I calculate the second moment, ie. the variance, $E[X^2]$ I get the value $3$. This seems to say that the distribution 'spreads' from $0$ to $3$.

Now if I calculate the variance about the mean, $E[(X - E[X])^2]$ I get the value $0.75$. What does this figure of $0.75$ mean? Intuitively I would have expected this value to be $1.5$, as in the distribution spreads from the mean value of $1.5$ to $0$, and to $3$, ie. it spreads out $1.5$ in each direction about the mean...so how do I interpret this seemingly arbitrary value of $0.75$?

  • $\begingroup$ The second moment is not the same thing as the variance. The variance is the central second moment, which is, as you write, $E\left[(X-E[X])^{2}\right]$. The "uncentered" second moment $E\left[ X^2\right]$ is something else. $\endgroup$ – Unwisdom Jan 31 '14 at 19:51
  • $\begingroup$ How does this sound to you: en.wikipedia.org/wiki/… $\endgroup$ – Hoda Jan 31 '14 at 19:53
  • 1
    $\begingroup$ You are right that the variance is a measure of how much the distribution spreads out. The range is another such measure. The two quantities (range and variance) don't tell you the same thing, but they are at least a little bit similar. It's a bit like the mean and the median. They aren't the same thing, but they measure aspects of a distribution that are somewhat related. I mention the range because of your comment "This seems to say that the distribution 'spreads' from 0 to 3." In fact, the range of this distribution is 3, and that is the thing telling you that it spreads from 0 to 3. $\endgroup$ – Unwisdom Jan 31 '14 at 19:56
  • 1
    $\begingroup$ The uncentered second moment is just the usual second moment: $E\left[X^{2}\right]$. Subtracting the mean from a random variable is sometimes called centering: the resulting variable has a mean of 0. The centered second moment is the (uncentered) second moment of $X-E[X]$. $\endgroup$ – Unwisdom Jan 31 '14 at 19:58
  • 1
    $\begingroup$ It doesn't. That's the coincidence. $\endgroup$ – Unwisdom Jan 31 '14 at 20:01

I find the variance easier to understand in terms of the standard deviation. The variance is basically "the average squared distance between the observations and the mean", but that is not a very intuitive measure of variation in the observations. If you take the square root of the variance, you will get the standard deviation, which is therefore "the average distance between the observations and the mean".

So the value of 0.75 is easier to interpret in terms of the std. deviation $\sqrt{0.75}\approx 0.87$. This number tells you that on average the observations are $0.87$ units away from the mean.

  • 4
    $\begingroup$ "The average distance between observarion and mean" would be mean absolute deviation (or $0$ if you consider sign), not standard deviation $\endgroup$ – Luis Mendo Jan 31 '14 at 20:24
  • $\begingroup$ I thought a standard deviation of $0.87$ would mean that 68% of the observations were within a distance of $0.87$ from the mean? $\endgroup$ – sonicboom Jan 31 '14 at 20:33
  • $\begingroup$ Luis: Checking your link, you seem to be right -- perhaps "a typical distance bewteen the observations and the mean" would have been a better description. $\endgroup$ – René B. Christensen Jan 31 '14 at 20:45
  • $\begingroup$ Sonicboom: Only if the distribution is normal. It won't necessarily be true for arbitrary distributions. $\endgroup$ – René B. Christensen Jan 31 '14 at 20:46
  • 1
    $\begingroup$ You could interpret standard deviation as a root mean squared deviation, which is another kind of "average" deviation. $\endgroup$ – David K Sep 17 '19 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.