4
$\begingroup$

If I want to calculate the sample variance such as below:

enter image description here

Which becomes: $\left(\frac{1}{n}\right)^2 \cdot n(\sigma^2)= \frac{\sigma^2}{n} $...

My question is WHY does it become $$\left(\frac{1}{n}\right)^2?$$ In other words, why does the $(1/n)$ inside the variance become $(1/n)^2$?

I've read that this is because:

When a random variable is multiplied by a constant, it's variance gets multiplied by the square of the constant.

Again, though, I want to know why?

I've looked in multiple sources but they all seem to gloss over this point. I want to visually see why this is done.

Could someone please demonstrate why the $1/n$ is squared using my example?


Update:

As @symplectomorphic points out in a comment under their answer, my confusion was the result of not realizing there was a difference between the variance of a set of data and the variance of a random variable.

  • See @symplectomorphic's other comment for an explanation of the difference.

@symplectomorphic's answer provides a good conceptual walkthrough, while user @Tryss's answer provides the correct mathematical explanation. Thanks to both of you!

$\endgroup$
  • $\begingroup$ The variance is a quadratic function. Just look at any of its definitions. $\endgroup$ – Qiaochu Yuan Mar 22 '16 at 4:17
  • $\begingroup$ Formatting tips here. $\endgroup$ – Em. Mar 22 '16 at 4:20
7
$\begingroup$

You have that

$$\text{Var}(aX) = E[(aX)^2]-(E[aX])^2 = E[a^2 X^2]-(aE[X])^2 $$

$$=a^2 E[ X^2]-a^2(E[X])^2 $$ $$= a^2( E[X^2]-(E[X])^2 ) = a^2 \text{Var}(X)$$

edit : or this one may be more basic (depending on your definition of variance)

$$\text{Var}(aX) = E[(aX-E[aX])^2 ] = E[(aX-aE[X])^2 ] $$

$$=E[a^2(X-E[X])^2 ] $$ $$= a^2E[(X-E[X])^2 ] = a^2 \text{Var}(X)$$

$\endgroup$
  • $\begingroup$ I don't follow. Could you put it into the context of my equation above? $\endgroup$ – theforestecologist Mar 22 '16 at 4:23
  • 1
    $\begingroup$ @theforestecologist : use my post with $a=1/n$ and $X = X_1+ \cdots + X_n$ $\endgroup$ – Tryss Mar 22 '16 at 4:25
  • $\begingroup$ Your edit makes more sense to me. $\endgroup$ – theforestecologist Mar 22 '16 at 4:30
  • $\begingroup$ @theforestecologist : no, my $X$ is $X_1 + \cdots + X_n$ $\endgroup$ – Tryss Mar 22 '16 at 4:33
  • $\begingroup$ Sorry just saw that :p $\endgroup$ – theforestecologist Mar 22 '16 at 4:33
4
$\begingroup$

Tryss's answer is correct. But you seem to need a more elementary illustration. Here it is, at least for the variance of sample data. (Your question is really about the variance of a random variable, but the point is the same.)

Take the two numbers $1$ and $3$. The mean of this set of data is 2. The variance is the average squared deviation from the mean. The deviations from the mean are $-1$ and $1$, so the squared deviations are $1$ and $1$, so the average squared deviation is $1$. Hence the variance of this set of data is 1.

Now look what happens when we multiply the dataset by 4. Our two numbers become 4 and 12. The mean is now 8. (This illustrates that when you multiply by a constant, the mean gets multiplied by that constant.) The deviations from the mean are $-4$ and $4$ (the deviations also get multiplied by the constant). Therefore the squared deviations are 16 and 16, so the averaged squared deviation is $16$. Hence the variance of this new set of data is 16.

Moral: when we multiplied our data by 4, the variance got multiplied by 16. This is totally unsurprising, because the variance is the average squared deviation. When you multiply your data by a constant, the deviations also get multiplied by that constant, so the squared deviations get multiplied by the square of that constant.

$\endgroup$
  • $\begingroup$ Ok the two of you helped me out. I get it now. My problem was that I always thought that Var = $\Sigma(x_i - X)^2 / n$ and not simply $(x_i - X)^2$. I guess the fomer would be called "population variance" and the latter is just ..."variance"? $\endgroup$ – theforestecologist Mar 22 '16 at 4:58
  • $\begingroup$ though your answer helped me realize this important issue, I'm going to accept Tryss's answer b/c it should be more helpful for other people later on. Please don't remove yours though! $\endgroup$ – theforestecologist Mar 22 '16 at 5:04
  • 1
    $\begingroup$ @theforestecologist: evidently you are still confused because your first comment makes no sense. the variance of a set of data is defined as $(1/n)\Sigma_{i=1}^n(x_i-\mu)^2$ (or possibly $1/(n-1)$ depending on how pedantic you want to be, but this makes no difference when $n$ is huge). I used this formula implicitly in my post: it just says to take the average of the squared deviations from the mean. (the variance of a random variable $X$ is different, defined in terms of expectations: $\text{Var}(X)=E[(X-E[X])^2]=E[X^2]-E[X]^2$.) $\endgroup$ – symplectomorphic Mar 22 '16 at 5:36
  • 1
    $\begingroup$ perhaps you are confused because you don't understand the difference between the variance of a set of data and the variance of a random variable. $\endgroup$ – symplectomorphic Mar 22 '16 at 5:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.