# Where does the guassian function/normal or bell curve come from?

I am confused as to where the function for the normal distribtuion comes from. Where does the e and pi come from? In my textbook I am presented with the function,but I am unsure about where it came from. I have spent hours on google and have yet to find a good proof that i can understand. Come someone provide a source that shows an elementary proof of the normal curve function.

-
Your question is not clear. You need proof of what? – user17762 Oct 21 '12 at 19:42
Where the function comes from. (1/s sqrt(2pi))e^(-x^2/(2s^2)) – Andrew Kudwitt Oct 21 '12 at 19:50

-
Did the Gaussian/normal function arise our of the binomial distribution? – Andrew Kudwitt Oct 21 '12 at 20:11

The Gaussian (normal or bell curve) arises, when you look at a huge collection of random variables and whose distribution you are not aware of.

You might want to look up Central Limit Theorem (CLT). The CLT explains why when you have a huge collections of random variables, they appear to follows the Gaussian.

A version of the CLT goes as follows.

Consider $n$ independently and identically distributed random variables, $\{X_1,X_2,\ldots,X_n\}$, with mean $\mathbb{E}(X_k) = \mu$ and variance $\mathbb{E}((X_k-\mu)^2) = \sigma^2 < \infty$. Now look at the sample mean $$S_n = \dfrac{X_1 + X_2 + \cdots + X_n}{n}$$ The central limit theorem states that $$\sqrt{n}(S_n - \mu) \to_{d} \mathcal{N}(0,\sigma^2)$$ as $n \to \infty$.

A proof for a weak version goes like this. Consider the characteristic function of the random variable $X$, say $\phi_X(t)$. Expanding using Taylor series about $0$, we get that $$\phi_{X-\mu}(t) = \phi_{X-\mu}(0) + \phi_{X-\mu}'(0) t + \dfrac{\phi_{X-\mu}''(0)}2 t^2 + \mathcal{O}(t^4/n^4) = 1 - \dfrac{\sigma^2}2 t^2 + \mathcal{O}(t^4/n^4)$$ The characteristic function for $(S_n - \mu)$ is $$\phi_{S_n - \mu}(t) = \phi_{X - \mu}(t/n)^n = \left( 1 - \dfrac{\sigma^2}2 t^2/n^2 + \mathcal{O}(t^4/n^4)\right)^n$$ Hence, the characteristic function for $Z_n = \sqrt{n}(S_n - \mu)$ is $$\phi_{Z_n}(t) = \phi_{\sqrt{n}(S_n - \mu)}(t) = \phi_{S_n - \mu}(\sqrt{n}t) = \left( 1 - \dfrac{\sigma^2}2 nt^2/n^2 + \mathcal{O}(t^4/n^2)\right)^n = \left( 1 - \dfrac{\sigma^2 t^2/2}n + \mathcal{O}(t^4/n^2)\right)^n$$ Hence, $$\lim_{n \to \infty} \phi_{Z_n}(t) = \lim_{n \to \infty} \left( 1 - \dfrac{\sigma^2 t^2/2}n + \mathcal{O}(t^4/n^2)\right)^n = \exp(-\sigma^2 t^2/2)$$

Note that the proof above is true for independent, identically distributed random variables arising from any nice probability distribution. With a little more effort, you can show that the convergence holds for independent (but not necessarily identically distributed) random variables. This is why the normal distribution is fundamental and appears everywhere in probability and statistics. The number $e$ appears since $$e = \lim_{n \to \infty} \left(1+\dfrac1n \right)^n.$$ The $\pi$ you are reffering to appears as a normalizing constant since $$\int_{-\infty}^{\infty} \exp(-x^2) dx = \sqrt{\pi}$$

-
Can you please explain the steps. I am just a beginner. – Andrew Kudwitt Oct 21 '12 at 21:51

In the early 18th century, Abraham de Moivre, a Frenchman, wrote a book about probability in Latin. Later he fled to England to escape the persecution of Protestants in France, and wrote an updated version in English, called The Doctrine of Chances. In one chapter he considered this problem: toss a fair coin 1800 times. What is the probability that the number of heads is between some particular number not much more than 900 and some number not much less?

An exact answer is the sum of $\dbinom{1800}{x} (1/2)^{1800}$ as $x$ goes through all the value in the specified range. This is horribly cumbersome to compute. De Moivre found a way to do it that approaches exactness as the number of coin tosses grows, and very accurate when it's 1800. He showed that the standard deviation of the probability distribution is $15\sqrt{2}=\sqrt{1800/4\ {}}$ and the expected value is $900$, and the distribution is well approximated by the Gaussian probability distribution with that expected value and that standard deviation. This has probability density $$\text{constant}\cdot e^{-(x-900)^2/(2(1800/4))}.$$ He found the value of the constant numerically. Later his friend James Stirling showed that it is $$\frac{1}{\sqrt{2\pi} \cdot15\sqrt{2}}.$$ (Warning to anyone wanting to make practical use of this: If you want to know $\Pr(885\le\text{number of heads}\le910)$, note that this is equal to $\Pr(884<\text{number of heads}<911)$ and, with the bell-shaped curve, find $\Pr(884.5<\text{the random variable}<910.5)$. This is a "continuity correction".)

Why $e$? I don't think I'd want to answer this without going through a general explanation of why $e$ arises in calculus. When I've tried to explain this to a class, they say things like "We don't need to know this to get an "A" in this course from other instructors!" and I think who cares? Doesn't that mean you get more for you money from me? To some students, understanding the material is the price they pay to get a grade; to better students, understanding the meterial is what they're there for. At any rate, an explanation of that would make this answer much longer. I've already posted on that topic in response to other questions here.

Why $\pi$? In other words, how do we know that $$\int_{-\infty}^\infty e^{-x^2/2} \, dx = \frac{1}{\sqrt{2\pi\ {}}}\ ?$$ I think that question's been posted here a few times, and answered.

-

I'm not aware of an elementary derivation of the functional form itself, but I can give you my favorite bit of intuition on where it comes from.

Suppose I told you I had a function $f : \mathbb{R} \to \mathbb{R}$, and that $f(0) = 0$. Based on this extremely limited information, I then ask you to find me $g(a \, | \, x)$, a probability density function for the value of $f(x)$.

There is one crucial observation you can make that allows you to find a partial solution. That observation is: for any $a, \Delta \in \mathbb{R}$, intuitively speaking, it must be true that $P(f(x) = a \, | \, f(0) = 0) = P(f(x + \Delta) = b + a \, | \, f(\Delta) = b)$. In other words, you have no reason to believe that the point $(0, 0)$ is special: you might as well rephrase the problem as what are the odds that $f$ increases by $a$ when its parameter is changed by $\Delta$? and assume that these odds are the same at any point of $f$.

We can use this to set up an integral equation. When $f$ travels from $(0, 0)$ to $(x, ?)$, imagine it first travelling to some intermediate point $x'$ and then on to $x$ from there. Then for any $x'$ between $0$ and $x$, it must be true that: $$g(a \, | \, x) = \int_{-\infty}^{\infty} \, g(a' \, | \, x') g(a - a' | x - x') \, da'$$

The solution to this integral equation is: $$\frac{1}{\sqrt{2\pi c_1x}} \, e^{-\frac{1}{2}\frac{(a - c_2 x)^2}{c_1 x}}$$

This equation is commonly referred to as the Gaussian with drift. $c_1 x$ is the variance and $c_2 x$ is the mean.

-
Where did the integral come from? – Andrew Kudwitt Oct 21 '12 at 21:23