7
$\begingroup$

I'm studying an introduction to probability, and before the explanation of the Central Limit Theorem the author presents the Gaussian Normal Distribution.

It has a complex and a non-intuitive formula, even if it's very important in the Probability field. Was the gaussian distribution found before the deduction of the CLT ? How was this distribution discovered ? And what are the properties that makes it so special ?

$\endgroup$
2
  • 2
    $\begingroup$ Good question(s) and don't mean to be rude, but sometimes simply searching Google will surely give you what you are looking for. $\endgroup$ Jul 2, 2018 at 8:40
  • 2
    $\begingroup$ The Wikipedia entry for the Normal distribution includes a History section. en.wikipedia.org/wiki/Normal_distribution#History $\endgroup$
    – awkward
    Jul 2, 2018 at 12:03

3 Answers 3

4
$\begingroup$

I want to comment on "It has a complex and a non-intuitive formula".

Let us first look at the reduced equation,

$$p(x)\propto e^{-x^2/2}.$$

This is simply the exponential of a square, nothing really difficult. The coefficient $2$ serves as a scaling factor to ensure that the standard deviation is $1$. (I won't give the technical details here as this involves improper integrals.)

Now we need to normalize the function so that the area under the curve is $1$ and the final formula is

$$p(x)=\frac1{\sqrt{2\pi}}e^{-x^2/2}.$$

If we consider general mean $\mu$ and standard deviation $\sigma$, the formula is simply adapted by applying the linear transform $x\to(x-\mu)/\sigma$ that shifts $\mu$ to $0$ and rescales $\sigma$ to $1$. To ensure that the area remains $1$, the normalization factor needs to be multiplied by $\sigma$ to compensate. We have

$$p(x)=\frac1{\sqrt{2\pi}\sigma}e^{-(x-\mu)/2\sigma^2}.$$

If you look at the shape of the function, is very smooth, it is symmetrical, it has a single maximum and quickly decays on both sides. This behavior describes a random variable that remains "concentrated" around the mean.

enter image description here


To better understand the shape, you can look at the distributions of the sums of $n$ uniform random variables in the range $0,1$, for increasing $n$:

enter image description here

You are smoothing and smoothing the curve, which quickly converges to a Gaussian.

Now observe that with $x+y=z$, where $x,y$ follow a reduced Gaussian

$$\int_{-\infty}^\infty e^{-x^2/2}e^{-y^2/2}dx=\int_{-\infty}^\infty e^{-x^2/2}e^{-(z-x)^2/2}dx=\int_{-\infty}^\infty e^{-(x-z/2)^2/2-z^2/4}dx \\=e^{-z^2/4}\int_{-\infty}^\infty e^{-x^2/2}dx\propto e^{-z^2/4}.$$

This proves that the distribution of the sum of two Gaussians is another Gaussian, with variance $2$. By a more complicated argument (based on the Fourier transform of convolutions), one can show the reciprocal: an additive distribution must be Gaussian.

$\endgroup$
3
  • $\begingroup$ An additive distribution must be Gaussian? I suspect you've forgotten some assumptions. $\endgroup$
    – J.G.
    Jul 2, 2018 at 10:39
  • $\begingroup$ @J.G.: of course, this is the "short version". $\endgroup$
    – user65203
    Jul 2, 2018 at 10:44
  • $\begingroup$ Where can i find the "complete version" ? $\endgroup$
    – Qwerto
    Jul 2, 2018 at 17:01
1
$\begingroup$

Speaking to the historical question, it was "discovered" from an attempt to work out what distribution a sample mean should have, and there were some false starts on the way, but I'll leave it there because this isn't HSM.

The Gaussian distribution isn't that non-intuitive, if you look at it the right way. If the log-pdf has a Taylor series and we expand around the mode (which can be taken as $0$ after a translation), the lowest-order term that can survive is an $x^2/2!$ term. The integral $\int_\mathbb{R}\exp -x^2/2\, \text{d}x$ was obtained in the late 1700s (the proof everyone knows today took a little longer; the early arguments, though valid, weren't as short or clean), and it wasn't much harder to verify that the pdf proportional to $\exp -x^2/2$ is of variance $1$, which is a nice normalisation. The "complex" formula you're thinking of is just a linear generalisation, viz. $f(x)\mapsto \tfrac{1}{\sigma}f(\tfrac{x-\mu }{\sigma})$. This is of course slightly neater if you work with CDFs first, viz. $F(x)\mapsto F(\tfrac{x-\mu }{\sigma})$.

There are several reasons Gaussian distributions can arise naturally, besides the CLT. These are discussed in my and others' answers here. My answer in particular discusses several problems to which a Gaussian is the unique or optimal solution.

$\endgroup$
1
$\begingroup$

Special and characteristic properties:

  • If $X$ has normal distribution then so has $aX+b$ where $a,b$ are constants with $a\neq0$.

  • If $X$ and $Y$ have joint normal distribution then so has $X+Y$.

Note that - if $X_n$ and $Y_n$ are sequences of random variables tending to Gaussians $X$ and $Y$ when $n\to\infty$ on base of CLT - also $X_n+Y_n$ will tend to a Gausian $Z$, so that $X+Y=Z$ where the LHS is a sum of Gaussians. This kind of explains why a summation of Gaussians is a Gaussian again, and tells us why Gaussian distribution is the only "candidate" for the CLT.

Similarly $aX_n$ will tend to a Gaussian, implying that $aX$ is Gaussian again.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .