7
$\begingroup$

I was going through this wikipedia article on standard error. I could not understand the crucial step here. It goes like this:

This formula may be derived from what we know about the variance of a sum of independent random variables.

If $X_1, X_2 , \ldots, X_n$ are n independent observations from a population that has a mean $\mu$ and standard deviation $\sigma$ , then the variance of the total

$T = (X_1 + X_2 + \cdots + X_n)$ is $n\sigma^2$. Understood.

The variance of T/n must be $\frac{1}{n^2}n\sigma^2=\frac{\sigma^2}{n}$. Not understood.

And the standard deviation of T/n must be $\sigma/{\sqrt{n}}$ . Of course, T/n is the sample mean $\bar{x}$ .

I went to some basics:

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}({x_i-\mu})^2$

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}({x_i^2+\mu^2-2x_i\mu})$

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}x_i^2+\mu^2-\frac{2}{n}\sum_{i=1}^{n}x_i\mu$

As Sample mean is an unbiased estimate of population mean, we get

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}x_i^2-\mu^2$

$\displaystyle Var(X)=E(X^2)-(E(X))^2$

Nothing useful found from this.

Why is there a $1/n^2$ in that step to get variance ?

$\endgroup$

3 Answers 3

6
$\begingroup$

Let $Y$ be any random variable. Let $Z = Y/n$. Then $$Z^2 = \frac1{n^2} Y^2,$$ $$E(Z^2) = E\left(\frac1{n^2} Y^2\right) = \frac1{n^2} E(Y^2)$$ and therefore $$E\left(\left(\frac Yn\right)^2\right) = \frac1{n^2} E(Y^2).$$ Also, $$E(Z) = E\left(\frac1n Y\right) = \frac1n E(Y).$$ So from $Var(Y)=E(Y^2)-(E(Y))^2$ and $Var(Z)=E(Z^2)-(E(Z))^2,$ we find $$\begin{eqnarray} Var\left(\frac Yn\right) = Var(Z) &=& E(Z^2)-(E(Z))^2\\ &=& \frac1{n^2} E(Y^2) - \left(\frac1n E(Y)\right)^2 \\ &=& \frac1{n^2} \left(E(Y^2) - \left( E(Y)\right)^2 \right) \\ &=& \frac1{n^2} Var(Y). \end{eqnarray}$$ Now consider the case where $Y = T$.

$\endgroup$
2
  • $\begingroup$ Why do we take Z = Y/n in the first place? Just to introduce the constant 1/n? I do not understand why we would want to introduce that constant in the first place. $\endgroup$
    – Saskia
    Nov 4, 2019 at 22:14
  • $\begingroup$ @Oleksandra The question that was asked above was essentially, why is it that when I divide a variable by $n$, the variance is divided by $n^2$ instead of just $n$? So this answer divides a variable by $n$ to show what happens. The reason why to divide by $n$ in the first place was evidently already understood. If you do not understand, you may ask a new question about that. $\endgroup$
    – David K
    Nov 4, 2019 at 23:06
3
$\begingroup$

The only thing we need to prove here is that for any scalar constant $c$, and for a random variable $X$, $$\mathrm{Var}[cX] = c^2 \mathrm{Var}[X].$$ This follows from the property of expectation $$\mathrm{E}[cX] = c\mathrm{E}[X]$$ as follows: $$\begin{align*} \mathrm{Var}[cX] &= \mathrm{E}[(cX - \mathrm{E}[cX])^2] \\ &= \mathrm{E}[(cX - c\mathrm{E}[X])^2] \\ &= \mathrm{E}[c^2(X - \mathrm{E}[X])^2] \\ &= c^2 \mathrm{E}[(X - \mathrm{E}[X])^2] \\ &= c^2 \mathrm{Var}[X]. \end{align*}$$

$\endgroup$
1
$\begingroup$

Recall the definition of (population) variance: $$var\xi := E(\xi - E\xi)^{2}$$ for $\xi$ a random variable. Then we have $var\xi = E\xi^{2} - 2(E\xi)^{2} + (E\xi)^{2} = E\xi^{2} - (E\xi)^{2}$ so that $var(\xi/n) = E(\xi^{2})/n^{2} - (E\xi)^{2}/n^{2}.$

Thus we have

$$var(T/n) := var(X_{1}/n) + \cdots + var(X_{n}/n) = n\sigma^{2}/n^{2} = \sigma^{2}/n.$$

$\endgroup$
4
  • $\begingroup$ What if $E(\xi) \neq 0$. Does this mean that an underlying assumption that population mean is zero is required for this formula to hold true ?I am not sure if I am missing something obvious here..but can't wrap my head around this $\endgroup$
    – square_one
    Aug 23, 2014 at 14:47
  • $\begingroup$ Let me revise the answer. Letting $E\xi := 0$ is to simply calculation. In your question, since $varX_{i}\ (i = 1, \dots, n)$ are given, there is no need to do these derivations from the great beginning. $\endgroup$
    – Yes
    Aug 23, 2014 at 14:49
  • $\begingroup$ using $\xi$ seems to just make this harder to read, is this symbol a convention? $\endgroup$
    – baxx
    Jan 11, 2020 at 15:24
  • $\begingroup$ @baxx, Hi, as far as I am aware, for older probabilists, especially for the Russian leading ones, symbols such as $\xi, \eta$ are more "standard"; a non-Russian example is Billingsley. $\endgroup$
    – Yes
    Jan 12, 2020 at 9:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .