1
$\begingroup$

I understand that the population variance of a set is just a special case of the variance of a random variable where $P(X=x) = \frac{1}{n}$ for $n$ elements in the set.

Still I can't help but feel that when computing $\sigma^2 (X) = \sum_{i=1}^n [(x_i -\mu)^2 \cdot P(X=x_i)]$, we're doing extra, or somehow "too much", individual weighing of the values of $X$.

After all, aren't the weights by likelihood of occurrence of a certain $x_i$ already expressed in $\mu$? Why are we again weighing the squared differences from the mean by probability? Why not average using $\frac{1}{n}$ instead?

$\endgroup$
  • $\begingroup$ Because then you'd be adding them instead of averaging them. The variance (loosely speaking) captures how much a variable may vary from its mean on average. $\endgroup$ – arctic tern Jul 3 '16 at 20:02
  • $\begingroup$ @arctictern, In the title of the question I ask about using $\frac{1}{n}$ instead. So I'm not saying don't average them the differences, I'm asking why average using the individual probabilities, since they're already expressed in $\mu$, and why not use $\frac{1}{n}$ instead? $\endgroup$ – jeremy radcliff Jul 3 '16 at 20:03
  • $\begingroup$ Why would we use $\frac{1}{n}$ if the different variations from the mean are not equally likely? $\endgroup$ – arctic tern Jul 3 '16 at 20:04
  • $\begingroup$ @arctictern I don't know, I just feel that the likelihood unevenness is already captured in the mean, so it feels like overkill somehow to express that unevenness yet again. $\endgroup$ – jeremy radcliff Jul 3 '16 at 20:05
  • $\begingroup$ Quote: "I'm asking why average using the individual probabilities ..." You don´t use the individual probability, but you divide by n. It is a coincindence that the individual probabilities are equal to $ \frac1n$. $\endgroup$ – callculus Jul 3 '16 at 20:10
2
$\begingroup$

The squared variation of $X$ from its mean could itself be considered a random variable, say defined by $Y=(X-\mu)^2$. The variation then measures the mean of $Y$, which involves using $Y$'s and hence $X$'s probability distribution. Your feelings wouldn't say to average a random variable (like $Y$) by simply adding the possible values it could take and dividing by the count if $Y$ is not uniformly distributed would they? If $x_1$ and $x_2$ are two values that $X$ can take with some probability, then the squared variation $(X-\mu)^2$ is not equally likely to be $(x_1-\mu)^2$ as it is to be $(x_2-\mu)^2$ unless for instance $X$ itself is equally likely to be $x_1$ as it is to be $x_2$.

Consider for instance a random variable $X$ that took the values $a$ or $b$ with probabilities $1$ and $0$ respectively. So essentially, $X$ always takes the value $a$, and in particular its mean is $\mu=a$.

According to your feelings, we shouldn't take into account how likely it is that $X$ is $a$ versus $b$ and simply calculate the average squared variation to be

$$\frac{1}{2}(a-a)^2+\frac{1}{2}(b-a)^2=\frac{(b-a)^2}{2}.$$

In other words, the average squared variation of $X$ from its mean would be $(b-a)^2/2$, even though $X$ never varies from its mean value $\mu=a$ and the value $b$ is completely independent of $X$ anyway, which is absurd. The issue is that the squared variation $(X-\mu)^2$ is equal to $(a-a)^2$ one-hundred percent of the time (not $\frac{1}{2}$ of the time) and is equal to $(b-a)^2$ zero percent of the time (not $\frac{1}{2}$ of the time), so on average $(X-a)^2$ should be

$$1(a-a)^2+0(b-a)^2=0.$$

$\endgroup$
  • $\begingroup$ Thank you for the added details. It took me a while to work through everything conceptually but it finally makes sense. $\endgroup$ – jeremy radcliff Jul 3 '16 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.