Why do bell curves appear everywhere?

Question

Why do most probability graphs show a bell curve? I've been wondering why... Is it just something natural, like the fibonacci sequence?

Answer 1

Why do most probability graphs show a bell curve?

As you suspect, there is a natural tendency for distributions to be bell-shaped.

There are some distributions that are not bell-shaped at all. For example, the outcome of a roll of one fair die is a discrete uniform distribution:

By IkamusumeFan - Own work This drawing was created with LibreOffice Draw, CC BY-SA 3.0, Link

The roll of one die is a pretty simple process. What about the sum of two dice? The Wizard of Odds illustrates:

Starting to look a little like a bell, right? What about the totals of three, or four dice? Wolfram MathWorld provides a nice illustration:

You can see where this is headed. Nature is full of complex processes. How tall are you? Well, it depends on genetics, nutrition, exercise, injuries, bone loss, and so many more things. The central limit theorem shows (see symplectomorphic's comment below) that when adding the sum of a large number of things together, the resulting distribution is not just any bell-looking curve, but specifically the normal distribution. Or for things with multiplicative combination, the log-normal distribution.

Why does this happen? mathreadler's answer hints it has to do with convolving distributions. The probability density function of a single die is a rectangular function (technically discrete, but let's pretend it's continuous). The sum of two rolls together is then the convolution of two rectangular functions.

By Convolution_of_box_signal_with_itself.gif: Brian Amberg derivative work: Tinos (talk) - Convolution_of_box_signal_with_itself.gif, CC BY-SA 3.0, Link

Notice how the result (the black triangle) looks like the case of two dice above. If then convolve this triangle with another rectangle, you get three dice. The more times you do this, the closer the result gets to a normal distribution.

The probability density function of the normal distribution is a Gaussian function, which have some elegant properties:

• A Gaussian convolved with a Gaussian is another Gaussian.
• The product of two Gaussians is a Gaussian.
• The Fourier transform of a Gaussian is a Gaussian.

From this you might intuitively see as things converge towards normal distributions, they "want" to stay as normal distributions since their "Gaussianess" is preserved under many operations.

Of course not everything is so simple as a single die roll, nor as complex as the determination of the height of a human. So there are a large number of distributions that look like a bell, but on careful examination aren't the normal distribution. Some of them exist in nature, and some find application as mathematical tools for some purpose. Looking through Wikipedia's list of probability distributions you can see bell-like shapes are quite common, even if they aren't exactly the normal distribution.

But if you combine these two things:

• The central limit theorem means the normal distribution is common, and
• many distributions look like bells but aren't the normal distribution,

you might conclude most probability graphs show a bell curve.

Answer 2

The convolution of two functions is at least as nice as the nicest of the two (often even nicer), and the sum of two independent distributions has a density which is the convolution of their density functions. So as they convolve more and more when we add them up they become nicer and the gaussian function is the nicest in the world!

Answer 3

I think it is important to distinguish between the general bell shaped curves that not have to be normal and the normal distribution. For the later, the key notion, as already mentioned and elaborated, is the Central Limit Theorem. Namely, if some slightly technical conditions are satisfied then sample averages and sums converge (weakly) to the normal distribution. However, $(1)$ not every bell shaped curve is normal and $(2)$ not everything that we assume that is normal is indeed normal. As already was mentioned in the comments - a lot of biological variables are definitely not normal (like heights and weights of humans) however they are bell shaped and can be approximated with very high precision with the Gaussian distribution. Same relation you can encounter with the Exponential distribution as a model for life duration of machine or something like this - they are definitely not really exponential as the machine cannot "live" forever.

As such, maybe a better model will be truncated distributions. E.g., heights maybe well described by two side truncated normal distribution. But what are the problems in this case? $(1)$ If the truncation values (parameters) are unknown, you have to estimate them, and $(2)$ besides it may introduce much more complexity to your calculations. So the basic questions in statistical modelling (IMHO) is not "whether this or that variable follows the normal distribution" albeit whether the Gaussian random variable can give us a good approximation of its distribution. Let us take the heights of healthy adult males in Scandinavia for example. A good model for their heights distribution will probably be $N(187, 10)$, but more accurate model will be the same Normal distribution but truncated at $5$ standard deviations above and below the mean, i.e., setting the support to be $[137, 237]$. Your gain in precision in estimating probabilities is absolutely neglectable as this truncation adds less than $0.001$ to the "mass" of the bell at $[137, 237]$. Same logic applies to the machine's life duration example. A truncation at $\tau$ that is far away from the expected value will give you correction term that is $1/P(X<\tau) = 1/(1-e^{-\lambda x})$, and it equals almost $1$, thus you have no practical gain from this.

Another issue that was already mentioned, is that asymptotic normality is not a universal truth. N.N. Taleb called the Gaussian distribution "the great intellectual fraud", where, to the best of my understanding, he meant that in finance the exponential decay ("thin" tails) is very uncommon feature. Namely, you can take Cauchy's distribution for example, it also has a bell shaped form, however due to relatively "fat" tails, it does not have a finite mean. This distribution will serve as a bad approximation of the heights of humans because negative heights or "gigantically" large values (let us say, over $4$ meters) are biologically impossible. Hence, a fat tail distribution that puts non-neglectable weights on extreme values will be improper mathematical modelling of such variable (height). While, in revenues it may be vice versa - assuming normality is the same as assuming neglectable probabilities for very large gains or losses, that - as we all know - maybe simply wrong.

To sum it up, it seems that symmetry and general "bell" shape is indeed prevalent in the real world distributions. However, strict normality as described by $$ f_{X}(x) = \frac{1}{\sqrt{2 \pi \sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}, $$
is mostly just an approximation of the actual distribution. More accurate models that are still "bell" shaped may be better in calculating various parameters, however the little gain from the more precise model usually don't worth the higher complexity that it introduces. Hence, the regular normal distribution remains in many cases not only a good approximation but also a very convenient one. Finally, it is worth to mention that are some "domains" like finance where the bell shaped distributions are mostly non-normal ones, and assuming normality in this case may be wrong.

Answer 4

As comment(s) suggest, the answer is the Central Limit Theorem

In probability theory, the central limit theorem (CLT) establishes that, in most situations, when independent random variables are added, their properly normalized sum tends toward a normal distribution (a bell curve) even if the original variables themselves are not normally distributed. The theorem is a key concept in probability theory because it implies that probabilistic and statistical methods that work for normal distributions can be applicable to many problems involving other types of distributions.

That suggests that natural independent variables on reasonable sample sizes tend to model what we call Normal Distributions, or gaussians.

Answer 5

I'll try to answer this with some intuition.

The "bell curve" (actually called the normal distribution) is one probability distribution that can be seen in the natural world. These are things like the distribution of heights and weights, where most people lie in the middle, with less occurrences on the higher and lower ends. How many 5-foot-6 people do you know? How about 6-foot-9? 4-foot-11?

As mentioned by @Chappers the "Central Limit Theorem" is one key explanation. Loosely speaking (very loosely speaking), this theorem says that if you repeat an experiment over and over, the distribution of averages will follow this normal distribution.

Answer 6

This is a question I've had myself, and from my experience talking with physicists, mathematicians, biologists, and engineers, their comment on a question like this was: a bell curve often represents a distribution of averages, which is to say mathematically, the distributions of averages of measurements always tend to form a bell-curve shape.

With this in mind, this shape naturally follows from the fact that most of our measurements are actually averages of many convoluted interactions on a smaller scale. "What is the blood pressure of a person?" Well blood pressure is an average of many averages of interactions with even more cells and chemicals. "What is the intelligence of a person?" Well, IQ is an average of many different culminations of cognitive interactions in the brain. "What is the temperature in in this room"? temperature is an average of the distribution of kinetic energy of many fast moving particles in an object or the air. Even though all of these things have many small fluctuations, they average out to the most common value with a bell distribution.

Answer 7

I found the bean machine to be the most intuitive example to justify the normal distribution. As rows are added to the bean machine the resulting distribution becomes an increasingly good approximation of the normal distribution. This is ultimately a consequence of the central limit theorem.

Notice the relationship with, say, coin flipping. If at each peg we go left on tails and right on heads then to get to the far left bin I'd have to have a long streak of tails to get there. By comparison, the central bins are more common because many different paths consisting of heads and tails will land you in the central bins unlike the outermost bins. Taking this binomial distribution to the limit as we increase the number of flips or rows in the bean machine and again we have the normal distribution. This is the most intuitive justification I've found.

Answer 8

Adding to the above answers, Gaussian functions also have nice computational properties: their Fourier transforms are Gaussian too, and Fourier transforms can be used to compute fast convolutions (like in one answer related to convolutions).

They can be used both as filters and windows with positive weights, and possess some optimal concentration in both domains, as famous from the Weyl-Heisenberg-Pauli so-called "uncertainty principle" (also SE.physics on Heisenberg/Gaussian waves).

It can also be found in the Azuma-Hoeffding inequality for martingales with bounded differences, and many others.

In vision, they form optimal scale-space structures, in relation with the heat equation.

However, they tend to be a little less efficient on (very) discrete data and outside least-square optimization.

Answer 9

The bell-curve, currently known as a normal distribution and formerly known as the exponential law of errors basically says that there is a nominal value and errors from this nominal value decrease in frequency the further you get from this nominal value.

Now the reason it is seen in nature comes from the variety of forces acting to generate an outcome. The distribution becomes normal when you have several different forces of varying magnitude acting together. Generally, the more forces then the more normal the distribution will become.

This occurs a lot in nature which is why the normal distribution is so prevalent. Say for example you are shooting a basketball, most likely you will at least hit the rim since you are the dominant force acting on the ball. However there are other forces as well: footing, wind, fatigue, elbow position, etc. So the main force dominates where the ball will go but many of the other forces have a small impact which generate variability.

Now in practice, the normal distribution is used more than it is observed in nature because it has several desirable qualities, is simple to understand, and it's shortcomings are often minute; so it serves as a good approximation.

It also plays a large role since most statistical estimators are ideally normal; so most methods strive for this sense a normal estimate is unbiased, centered, with errors decreasing in frequency the further from nominal.

Answer 10

Heat is random motion. The rate of flow of random motion behaves like the flow of heat.

The rate of change of heat at a position is the rate of heat going in minus the heat going out. The rate of heat transfer is proportional to the difference in temperature. In one dimension,

$$\frac{du}{dt} = k \frac{d^2u}{dx^2}$$

Suppose you have a spike of heat. All the heat in one place. $$u(x,0)=\delta(x)$$ where $\delta$ is the Kronecker delta function.

With $\sigma^2 = kt$ and $\mu=0$, the normal (Gaussian) distribution is a solution to this initial value problem. $$\Phi(x,t)=\frac{1}{\sqrt{4\pi kt}}\exp\left(-\frac{x^2}{4kt}\right)$$

Over time the heat will fall down into a bell curve. This gives an intuitive idea of how randomness flows away from an initial exact position towards the two tails of the bell curve.

Answer 11

While many of the explanations above are quite excellent and bring to light the CLT and convergence of distributions to normal shaped things, there is a point that I think needed to be emphasized (and perhaps add a different viewing angle).

When first reading this question, the answer I came up with did not land immediately on the CLT. You can break the question down into two parts:

1. Why do the functions taper off at the end?
2. Why are the functions unimodal (only a single hump in the middle)? Why is it symetric?

To answer the first question, I draw on properties of distributions in general. Specifically the the normality requirement of the PDF $\int_{-\infty}^{\infty} f(x)dx = 1$. For this requirement to hold, a sufficient (but not necessary) condition is that $lim_{x\rightarrow\infty}f(x)=0$ and $lim_{x\rightarrow-\infty}f(x)=0$. This means you will very often end up with curves that are "fat" in the middle and taper off at the ends. It is an natural consequence of the normality condition that most of the "interesting" stuff in a distribution happens in the middle (but of course there are degenerate counter examples to this sweeping rule).

The answer to second question is exactly the CLT as others have pointed out. But lets take it for a different spin. Suppose you have a uniform distribution on the interval $(0,1)$. Begin "drawing" from this distribution, and noting the values you get when you do. One big observation about this distribution is that, necessarily, $\frac{1}{2}$ of the probability, lies below the $\frac{1}{2}$ way mark ($0.5$ in this case). Said another way, it's an artifact of the distribution that, you are equally likely to get a value above the $\frac{1}{2}$ way mark as below it. So as you "draw" from the distribution, the value of the samples will naturally tend to balance them selves out. For example note that $P(X<.25)=P(.75<X)$ for uniformly distributed $X$. Thus you'll end up with a hump in the middle (thats the key behind all the other posts about Galton's bean's).

So why bring up the uniform distribution? In a lot of cases, when you want to model a starting assumption of zero information (an uninformed model), you start with some "reasonable" bounds and assume that the values are uniformly distributed. Then you sample and update the model. The updated model will have some symmetry because of the property I just mentioned. If you want to cut out a "large bin" of probability this symmetry will force you to do it from the middle. Hence, the unimodal behavior.

Is it "Gaussian"? Not necessarily as some other posters have mentioned. Long tails can occur under the right conditions (e.g. if you start off with stuff that is not completely uniform but more $\beta$ shaped with a flat middle but high but short "ends"). But is "bell shaped", yes.

Answer 12

A bell curve is a Gaussian distribution curve. It occurs very frequently. Like a pile of sand you slowly let fall from your hand on the beach, the shape of the pile, in one plane, is such a curve. It's everywhere from the beach to the atom.

But with all this theorising above, don't lose sight of the fact that mathematics is merely a human construct that reflects the way that matter and spacetime interact with each other and themselves. Mathematics does not dictate the way the Universe behaves; it merely reflects it.