4
$\begingroup$

I was going through this wikipedia article on standard error. I could not understand the crucial step here. It goes like this:

This formula may be derived from what we know about the variance of a sum of independent random variables.

If $X_1, X_2 , \ldots, X_n$ are n independent observations from a population that has a mean $\mu$ and standard deviation $\sigma$ , then the variance of the total

$T = (X_1 + X_2 + \cdots + X_n)$ is $n\sigma^2$. Understood.

The variance of T/n must be $\frac{1}{n^2}n\sigma^2=\frac{\sigma^2}{n}$. Not understood.

And the standard deviation of T/n must be $\sigma/{\sqrt{n}}$ . Of course, T/n is the sample mean $\bar{x}$ .

I went to some basics:

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}({x_i-\mu})^2$

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}({x_i^2+\mu^2-2x_i\mu})$

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}x_i^2+\mu^2-\frac{2}{n}\sum_{i=1}^{n}x_i\mu$

As Sample mean is an unbiased estimate of population mean, we get

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}x_i^2-\mu^2$

$\displaystyle Var(X)=E(X^2)-(E(X))^2$

Nothing useful found from this.

Why is there a $1/n^2$ in that step to get variance ?

$\endgroup$
2
$\begingroup$

Let $Y$ be any random variable. Let $Z = Y/n$. Then $$Z^2 = \frac1{n^2} Y^2,$$ $$E(Z^2) = E\left(\frac1{n^2} Y^2\right) = \frac1{n^2} E(Y^2)$$ and therefore $$E\left(\left(\frac Yn\right)^2\right) = \frac1{n^2} E(Y^2).$$ Also, $$E(Z) = E\left(\frac1n Y\right) = \frac1n E(Y).$$ So from $Var(Y)=E(Y^2)-(E(Y))^2$ and $Var(Z)=E(Z^2)-(E(Z))^2,$ we find $$\begin{eqnarray} Var\left(\frac Yn\right) = Var(Z) &=& E(Z^2)-(E(Z))^2\\ &=& \frac1{n^2} E(Y^2) - \left(\frac1n E(Y)\right)^2 \\ &=& \frac1{n^2} \left(E(Y^2) - \left( E(Y)\right)^2 \right) \\ &=& \frac1{n^2} Var(Y). \end{eqnarray}$$ Now consider the case where $Y = T$.

$\endgroup$
  • $\begingroup$ Why do we take Z = Y/n in the first place? Just to introduce the constant 1/n? I do not understand why we would want to introduce that constant in the first place. $\endgroup$ – Saskia Nov 4 '19 at 22:14
  • $\begingroup$ @Oleksandra The question that was asked above was essentially, why is it that when I divide a variable by $n$, the variance is divided by $n^2$ instead of just $n$? So this answer divides a variable by $n$ to show what happens. The reason why to divide by $n$ in the first place was evidently already understood. If you do not understand, you may ask a new question about that. $\endgroup$ – David K Nov 4 '19 at 23:06
2
$\begingroup$

The only thing we need to prove here is that for any scalar constant $c$, and for a random variable $X$, $$\mathrm{Var}[cX] = c^2 \mathrm{Var}[X].$$ This follows from the property of expectation $$\mathrm{E}[cX] = c\mathrm{E}[X]$$ as follows: $$\begin{align*} \mathrm{Var}[cX] &= \mathrm{E}[(cX - \mathrm{E}[cX])^2] \\ &= \mathrm{E}[(cX - c\mathrm{E}[X])^2] \\ &= \mathrm{E}[c^2(X - \mathrm{E}[X])^2] \\ &= c^2 \mathrm{E}[(X - \mathrm{E}[X])^2] \\ &= c^2 \mathrm{Var}[X]. \end{align*}$$

$\endgroup$
1
$\begingroup$

Recall the definition of (population) variance: $$var\xi := E(\xi - E\xi)^{2}$$ for $\xi$ a random variable. Then we have $var\xi = E\xi^{2} - 2(E\xi)^{2} + (E\xi)^{2} = E\xi^{2} - (E\xi)^{2}$ so that $var(\xi/n) = E(\xi^{2})/n^{2} - (E\xi)^{2}/n^{2}.$

Thus we have

$$var(T/n) := var(X_{1}/n) + \cdots + var(X_{n}/n) = n\sigma^{2}/n^{2} = \sigma^{2}/n.$$

$\endgroup$
  • $\begingroup$ What if $E(\xi) \neq 0$. Does this mean that an underlying assumption that population mean is zero is required for this formula to hold true ?I am not sure if I am missing something obvious here..but can't wrap my head around this $\endgroup$ – square_one Aug 23 '14 at 14:47
  • $\begingroup$ Let me revise the answer. Letting $E\xi := 0$ is to simply calculation. In your question, since $varX_{i}\ (i = 1, \dots, n)$ are given, there is no need to do these derivations from the great beginning. $\endgroup$ – Megadeth Aug 23 '14 at 14:49
  • $\begingroup$ using $\xi$ seems to just make this harder to read, is this symbol a convention? $\endgroup$ – baxx Jan 11 at 15:24
  • $\begingroup$ @baxx, Hi, as far as I am aware, for older probabilists, especially for the Russian leading ones, symbols such as $\xi, \eta$ are more "standard"; a non-Russian example is Billingsley. $\endgroup$ – Megadeth Jan 12 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.