0
$\begingroup$

Central Limit Theorem (Sum Version)

Let $X$ be a population random variable with finite mean $\mu_X$ and finite variance $\sigma^2_X$ and let $(X)_{i=1}^n$ be a random sample of $X$. Let $S= \displaystyle\sum_{i=1}^n X_i.$ Then for large sample sizes $n$ the distribution of $S$ is approximately normally distributed with mean $\mu_S = n \cdot \mu_X$ and variance $\sigma_S^2= n \cdot \sigma_X^2.$

What this part of the theorem says is this, if we take $n$ sample sizes from the population, the distribution is approximately normally distributed. Another words as $n$ grows larger, it converges to the normal distribution. The theorem is best used when you are taking $n>30$ sample size from the population. Since the sample size is approximately normally distributed, we can standardize the random variables so that we can estimate what percentage of values of the sample mean fall within some given interval. For example, if I have a population of 100,000 and I take a survey of 30 people to see what they think about math. Than we have $(X_i)_{i=1}^{30}$ where each $X_1$ represents the outcome of the 1st person, $X_2$ the second, $X_{30}$ the 30th person etc.

Central Limit Theorem (Sample Mean Version)

Let $X$ be a population random variable with finite mean $\mu_X$ and finite variance $\sigma_X^2$ and let $(X)_{i=1}^n$ be a random sample of $X$. Let $\bar{X}=\dfrac{1}{n}\displaystyle\sum_{i=1}^n X_i$ denote the sample mean. Then for large sample sizes $n$, the distribution of $\bar{X}$ is approximately normal with mean $\mu_{\bar{X}}=\mu_X$ and variance $\sigma^2_X= \dfrac{\sigma_X^2}{n}$.

This other part of the theorem is what is confusing me. I cant see the connection of the mean and variance like the other one. I think this one is when we want to consider the proportion of values rather the other one is the sum of values. But I don't understand why the variance and mean change. Also why is the mean for the 1st part of the theorem $n \cdot \mu_X$? You would think that the mean would simply be the sum of the outcomes divided by the number of observations which isn't the case in the 1st part of the theorem.

$\endgroup$
3
  • 2
    $\begingroup$ We have $X_i$ is the result of the $i$th observation. In particular, each $X_i$ is a random variable, and therefore so is $(X_1+\cdots +X_n)/n$. If you do the experiment $n$ times, then do it again and again, the values of $(X_1+\cdots +X_n)/n$ will vary. We are calculating the mean and variance of this random variable. $\endgroup$
    – André Nicolas
    Mar 3, 2014 at 2:12
  • $\begingroup$ Oh I see we are considering the mean itself as a random variable. That makes sense because as $n$ gets large the variance converges to 0 which implies that this is a normal distribution $\endgroup$
    – adam
    Mar 3, 2014 at 2:19
  • $\begingroup$ The variance converges to $0$, but that's not why we have a near normal. If we shift and scale to make the mean $0$ and the variance $1$, the distribution becomes nearly standard normal. $\endgroup$
    – André Nicolas
    Mar 3, 2014 at 2:30

1 Answer 1

Reset to default
3
$\begingroup$

Take the first part and use the linearity property of the expected value, i.e. that $$E[aX+bY]=aE[X]+bE[Y]$$ and the property of the variance, that $$Var(aX+bY)=a^2Var(X)+b^2Var(Y)$$ for independent random variables $X,Y$.

So, you have that $$S_n:=\sum X_i \sim N(\mu_{S_n}=n\mu, \sigma^2_{S_n}=n\sigma^2)$$ and therefore if we consider the r.v. $\bar{X}_n=\frac{1}{n}S_n$ we get that $$E[\bar{X}_n]=\frac{1}{n}E[S_n]=\frac{1}{n}n\mu=\mu$$ and $$Var(\bar{X}_n)=\left(\frac{1}{n}\right)^2Var(S_n)=\left(\frac{1}{n}\right)^2n\sigma^2=\frac{1}{n}\sigma^2$$ Therefore $\bar{X}_n \sim N(\mu,\frac{\sigma^2}{n})$ where we also used the invariance of the normal distribution under linear transformations.

So the second version is derived from the first version through the linear transformation $S_n \rightarrow \frac{1}{n}S_n=:\bar{X}_n$! The intuition is however different.

$\endgroup$
8
  • $\begingroup$ Thanks this clears up the ambiguity I was having. I understood where they were coming from in the theorem I just didn't understand how the formulas were there. $\endgroup$
    – adam
    Mar 3, 2014 at 2:22
  • $\begingroup$ Ok, you are welcome. If you have other ambiguities, tell me $\endgroup$
    – Jimmy R.
    Mar 3, 2014 at 2:23
  • $\begingroup$ Ok so for the first part when we take $$E[S]=E[X_1]+E[X_2]+\cdots E[X_n]=n \cdot \mu_X$$ why do we assume that $E[X_1]=E[X_2]=\cdots E[X_n]$? and how do we know it is $\mu_X$ $\endgroup$
    – adam
    Mar 3, 2014 at 2:41
  • $\begingroup$ Assume you have a population $X$ with mean $\mu_X$ (for example the income). Then you sample from that population and denote by $X_1$ the answer of the first person (as you have already mentioned). What is his distribution, i.e. what are the probabilities that he will have each specific income? They are distributed exactly as $X$, in particular $E[X_1]=E[X]$. Now, you ask a second person. He has the same distribution. That is $X_2$ and so on. $\endgroup$
    – Jimmy R.
    Mar 3, 2014 at 2:52
  • $\begingroup$ Or take the other example. You produce identical computers. Then their lifetime is a random variable $X$ with mean $\mu_X$. The first pc has lifetime $X_1$ with mean $\mu_X$, the second pc has lifetime $X_2$ with mean $\mu_X$ and so on... They are copies of the same thing, that is the meaning of identically distributed. Ah, and of course they are independent. The lifetime of the one pc does not affect the lifetime of the others $\endgroup$
    – Jimmy R.
    Mar 3, 2014 at 2:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .