2
$\begingroup$

I understand the assumptions underpinning and the motivation behind error distributions which led to the development of Laplace's and eventually Gauss's error distributions as explained in Saul Stahl's essay. It makes sense that larger random errors are less likely than smaller ones, and that the magnitudes of errors are the same regardless of direction, i.e. $\phi(x) = \phi(-x)$.

What I'm having a hard time reconciling is the widespread use of the normal distribution to describe generic data, not just errors. Why can we model human heights or test grades with a normal distribution?

I understand The Central Limit Theorem allows us to use the normal distribution to approximate certain sampling distributions, e.g. $\overline{x}$, as normal (usually when $n \geq 30$) - but why? What does an error curve have to do with CLT? Do random factors that affect natural phenomena to cancel each other out leading to (approximately) normal distributions with large samples?

$\endgroup$
0
$\begingroup$

Are you clear what the "Central Limit Theorem" says? There doesn't have to be any "inherent" "normally distributed errors". The Central Limit Theorem say that given any probability distribution with a finite mean, $\mu$ and standard deviation, $\sigma$, then the sum of n samples will be approximately normally distribution with mean $n\mu$ and standard deviation $\sqrt{n}\sigma$ while the average of n samples is approximately distributed with mean $\mu$ and standard distribution $\frac{\sigma}{\sqrt{n}}$.

The larger n is, the better the approximation is. Generally n greater than or equal to 30 is considered enough to make the approximation "good".

$\endgroup$
0
$\begingroup$

Great question! I will be following this post. While I don't fully answer the question, here are some thoughts on why CLT is so prevelant.

I am actually writing a blog on this very thing but it is under construction. How it goes is that once you subtract the mean from a distribution, you are left with the deviations from the mean, and the expected value of this is 0 regardless from the distribution it came from. Now every time I sample from this distribution of deviations I get values to the left and right as you described creating a normal distribution.

A quote the blog : Any process that adds together random values from the same distribution converges to a normal. But it’s not easy to grasp why addition should result in a bell curve of sums. Here’s a conceptual way to think of the process. Whatever the average value of the source distribution, each sample from it can be thought of as a fluctuation from that average value. When we begin to add these fluctuations together, they also begin to cancel one another out.

It does this because there are so many more possible ways to realize a sequence of positive negative steps that sums to zero. There are slightly fewer ways to realize a sequence that ends up plus or minus one, and so on, with the number of possible sequences declining in the characteristic bell curve of the normal distribution.

The blog is like 70% complete but the first half or so is quite readable. You can check it out if you are interested :https://bluesky314.github.io/Coming-Soon-Guessing-the-Central-Limit-Theorem/

$\endgroup$
  • $\begingroup$ "When we begin to add these fluctuations together, they also begin to cancel on another out". But based on a certain distribution, skew can affect the relative likelihoods of values, i.e. a skewed left distribution favors larger values. Is it not true that when we sample many times that we will more likely have greater values? How would they cancel out? I've been taught the CLT but never been taught the intuition behind it. $\endgroup$ – Andrew Li Jul 9 '19 at 18:59
  • $\begingroup$ No, because we are now considering the mean-subtracted distribution. And you are not taking the frequencies into account. I gave an example of skewed Beta distribution in the blog with images. "Thats because there are a few large deviations to the right but many smaller deviations to the left. When we sum them, the smaller values occur with a much higher frequency so they are able to cancel the larger values. So once again, the mean is the point around which the positive and negative deviations cancel out." $\endgroup$ – Rahul Deora Jul 10 '19 at 5:10
0
$\begingroup$

We model them with the normal distribution because it is convenient, not because it is correct. For real distributions, the tails are usually much larger than the normal distribution would have them if you assess the standard deviation based on most of the numbers. There is a quote attributed to Poincare that has been deleted from the Wikipedia page to the effect that "Everyone believes the normal distribution-physicists because they think it is a mathematical theorem, mathematicians because they believe it is an experimental fact."

$\endgroup$
  • $\begingroup$ So there is no other mathematical motivation other than it's experimentally verified? If the normal distribution was derived by understanding random errors, what part does that play in CLT? Feels kind of weird how a large enough sample can yield a normal distribution without attribution to random errors which form the basis of the distribution. $\endgroup$ – Andrew Li Jul 9 '19 at 18:50
  • $\begingroup$ It is not an experimental fact, either. It is a convenient approximation that works well as long as you don't go too far out on the tails, where it will usually claim that events are much rarer than they are in practice. $\endgroup$ – Ross Millikan Jul 9 '19 at 18:56
  • $\begingroup$ Okay. I think my real qualm is about the CLT and how it is related to random errors, since the CLT allows for approximations to be made with the normal distribution (which was originally an error curve). $\endgroup$ – Andrew Li Jul 9 '19 at 19:02
  • $\begingroup$ The CLT says that if you add up enough measurements, regardless of their individual distributions, the result will be approximately normal. However, think of adding ten measurements taken with a tape measure. Maybe you measure to $\pm \frac 1{16}$ inch. You would then expect the standard deviation of the sum to be around $\frac 3{16}$ inch. A $1$ inch error in the sum presumably has zero probability, but we all know that it happens. $\endgroup$ – Ross Millikan Jul 9 '19 at 19:33
0
$\begingroup$

You are obviously referring to the Central Limit Theorem which can be applied for a large enough sample from a non-normally distributed population.

It is a remarkable theorem which essentially says:

If all possible random samples of size $n$ are taken from $any$ population having mean $\mu$ and standard deviation $\sigma$, then the distribution of sample \underline{means} will:

(1) have a mean ($\mu_{x}$) which is equal to $\mu$.

(2) have a standard deviation $(\sigma_{x})$ that is equal to $\sigma$

(3) be normally distributed provided that the underlying population is normally distributed; otherwise, it will be at least approximately normally distributed for sample sizes of $n \geq 30$; and, the approximation to normality will improve as the sample size increases.

Thus, this theorem allows us to relate the mean $\bar{x}$ of our sample to the "at least approximately normally distributed" distribution of sample means whether or not our population is normally distributed provided that the sample size of our random sample is $30$ or more.

$\endgroup$
  • $\begingroup$ But is there any insight gained when factoring in random errors? Is there any intuitive explanation why CLT works? In my mind if there is it should involve random errors which is how the normal distribution is derived. $\endgroup$ – Andrew Li Jul 9 '19 at 18:55
  • $\begingroup$ Intuitive reasons do not supplant a proof; that being said, I doubt that you are to find many (or any) undergraduate introductory statistics texts which includes the proof of the CLT. Hence, try as best you can to cultivate an intuition for what the theorem says if you are enrolled in an introductory course. $\endgroup$ – mlchristians Jul 9 '19 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.