# What gives rise to the normal distribution?

I'd like to know if anyone has a generally friendly explanation of why the normal distribution is an attractor of so many observed behaviors in their eventuality. I have a degree in math if you want to get technical, but I'd like to be able to explain to my grandma as well

-
I don't know how friendly you expect the explanation to be. The PDF itself is not very natural to my grandma. The most natural explanation in my opinion is the central limit theorem. –  Tunococ Oct 17 '12 at 6:02
Take her to the Museum of Science to see one of these! en.wikipedia.org/wiki/Bean_machine –  Qiaochu Yuan Oct 17 '12 at 6:13
As @Tunococ says. Check the Wikipedia article on the central limit theorem (this part‌​) to get a pretty good idea how the normal distribution arises. –  Harald Hanche-Olsen Oct 17 '12 at 6:37

I'm not sure if this is good for your grandma, but I'll give it a try.

Normal distribution here is meant to be the one symmetric around the mean (which is also the mode, most frequent observation), i.e. with 0 skewness.

Normal distribution arises as a limiting case of many other distributions with the increase of the size of the sample, e.g. Binomial or Poisson due to Central Limit theorem (hence limit) when you standardize this distribution around the mean, $Z=\frac{X- \mu}{\sigma}$ (hence central). In essence, people noticed that different unrelated distributions have a very similar convergence pattern if you standardize them in this way.

It also arises naturally in many aspects of real life, since many phenomena have some tendency to symmetry around the mean. For example (I think it was one of Fisher's works) if you look at all people with weight=$m$ and rank them in the order of increasing heights, the frequency of heights forst increases, reaches a peak and then decreases, so if you look at the histogram of heights, it resembles (as the sample size keeps growing) a bell-shaped structure. Same can be said about many other processes/phenomena. Their frequency first grows, then peaks around the mean, then reduces.

So people decided to give it a name, Normal distribution.

-

A normal distribution arises whenever there are many small independent factors contributing more or less equally. A distribution which is not normal typically indicates that something interesting is happening, for example some random phenomena contributes much more than the others.

-

To my mind the reason for the pre-eminence can at best be seen in what must be the most electrifying half page of prose in the scientific literature, where Clark Maxwell deduces the distribution law for the velocities of molecules of an ideal gas (now known as the Maxwell-Boltzmann law), thus founding the discipline of statistical physics. This can be found in his collected papers or, more accessibly, in Hawking's anthology "On the Shoulders of Giants". The only assumptions he uses are that the density depends on the magnitude of the velocity (and not on the direction) and that the components parallel to the axes are independent. Mathematically, this means that the only functions in three-dimensional space which have radial symmetry and split as a product of three functions of the individual variables are those which arise in the normal distribution.

-
"Clark Maxwell"? –  Jonathan Oct 21 '12 at 19:35

When you plot the computed probabilities of getting $k$, $\ 0\leq k\leq 10$, heads in $n=10$ throws of a coin you obtain a histogram showing a very crude approximation of a bell curve. When you do the same thing for $n=100$ or $n=1000$ throws you get finer histograms of a similar shape, but the peak in the middle gets definitely sharper as $n$ increases. We see here the law of large numbers at work. Doing the calculations one finds that the relative width of the peak decreases proportionally to ${1\over\sqrt{n}}$. Compensating for this effect using suitable scalings one indeed obtains a unique limiting curve, the so called Gauss curve.

Now for the universality of this curve. When doing our histograms, a tail counted $0$ and a head $1$. So the result of the $i$th throw can be viewed as a random variable $X_i$ taking the values $0$ or $1$ each with probability ${1\over2}$. What the histogram shows is nothing else but the distribution of the sum $S:=X_1+X_2+\ldots+X_n$. (This might be a little difficult to explain to your grandma.) Up to a horizontal translation and a scaling the histograms would look exactly the same if the variables $X_i$ assumed the values $\pm1$ or $\pm0.001$ instead of $0$ and $1$.

Imagine now a physical experiment, say the measurement of the temperature $T$ at Kennedy airport on 21.12.2012 at 12:00 p.m., which in our long time experience should turn out the value $0$ ($0^\circ$ C in the case of our example). In reality the exact reading of $T$ is determined by a thousand tiny influences which act independently of each other;, but each of them is contributing $\pm0.001^\circ$ to the resulting value of $T$. Imagine now that we are doing this experiment for 50 years in a row and make a histogram of the measured $T$-values. We shall see our bell curve again.

-

I did not study mathematics beyond my junior year in high school, so I'm well-qualified to explain the whats and whys of the normal distribution in basic 'lay man' terms.

Step 1 - Understand what the mean represents on a distribution

Think of the mean as a mid-point in a multi-way tug-of-war between all the observations (or 'data points' or whatever you call them). The greater/smaller the size of each observation relative to the other data points, the greater its effect on the mean.

In any distribution, the area under the curve to the left of the mean is exactly equal to the area to the right of the mean. To understand this, it is useful to think of the mean as a 'weighted average' of all the data points - remember we are taking frequency into account too. Take a look at at any histogram and imagine each bar as stacks of lego bricks. If you re-arranged the bars into two big blocks on each side of the mean, they would look exactly the same (and the area would be exactly the same) as the mean is the 'compromise' between all the data points.

Therefore the probability of having ANY one of all the observations on the LEFT side of the mean is exactly the same as having ANY one of all the observations on the RIGHT side of the mean. This is very important for understanding the Central Limit Theorem.

Step 2 - Understand that data points/observations are themselves combinations of random factors

Think of each observation as the result of the interplay of a set of random, independent factors (as in the temperature example described by Christian Blatter on this thread). Each observation is basically the result of the combined effects of the independent factors pulling the observation higher or lower than the mean.

As a result, the probability of having a set of random factors that come together to create observations that are much higher/lower than the mean gets smaller and smaller the further away from the mean we go. This is because we need an increasing amount of combinations of variables that pull the observations in the same direction the further way from the mean we go.

Remember that the probability of having ANY observation on either side of the mean is exactly the same? We can extend this logic to say that the probability of ANY set of random factors ocurring to create an observation on either side of the mean is exactly the same.

As a result, it is much easier (and more likely) to have a combination of factors that create an observation somewhere around the mean (i.e. some above the mean, some below the mean, cancelling each other out). This is because there are a greater number of possible combinations that can create the mean than there are combinations that can create extreme 'tail end' values.

Step 3 - a simple example

You toss a coin twice and take the mean. Think of the mean as the observation and either side of the coin as the random variable/factor that determines the magnitude of the observation (as described above).

Each outcome has the following value:

Given a 50 per cent probability of either outcome, the mean is 1.5.

However, the distribution of outcomes will converge towards the mean. This is because there is a higher probability of getting combinations that result in the mean than combinations that do not:

Heads 1 (p = 0.5), Heads 1 (p = 0.5) = Outcome 1 (p = 0.25)

Heads 1 (p = 0.5), Tails 2 (p = 0.5) = Outcome 1.5 (p = 0.25)

Tails 2 (p = 0.5), Heads 1 (p = 0.5) = Outcome 1.5 (p = 0.25)

Tails 2 (p = 0.5), Tails 2 (p = 0.5) = Outcome 2 (p = 0.25)

As you can see the probability of getting the mean (1.5), in this very basic example, is twice that of getting 2 OR 1. Draw a graph, and you'll see the beginning of a very rudimentary normal-ish distribution. What do you think this would look like if you repeated the same exercise with a 6-sided die?

Hope this helps anyone looking for a non-mathematical way understanding the subject.

-