1
$\begingroup$

I have come across a proof of the Gaussian pdf that starts like a story - throwing darts on a Cartesian-ized target with $x$ and $y.$ The errors (it's all about the errors) will be independent in both directions and rotationally invariant.

From there, what follows is predictable - a change to polar coordinates, and integration over the circle, and in comes $\pi.$

Yes, this question spawns from this post on pi and the exponential that happened to be active today.

So in this dart set up we really start off from an uncorrelated bivariate normal to arrive at the pdf of the univariate normal.

Needless to say, the darts, the circle and pi provide with a wonderful intuition, but the question (understanding that there are more than one derivation of the Gaussian) is:

Why do we need to resort to the bivariate normal as a set up to arrive at the normal unvariate Gaussian pdf?

Is it purely practical? If so which step in the derivation necessitates this set up?

It is clear that the sooner we get to Pythagoras the smoother the path will be... Which reminds me of a recent tweet by @fermatslibrary that just starts point blank with the square of the integral:

enter image description here

$\endgroup$
1
  • $\begingroup$ Your question is a little rambling. Are you asking (i) if it's possible to obtain the form of the Gaussian PDF without considering a bivariate Gaussian, (ii) if it's possible to obtain specifically the normalization constant $\sqrt\pi$, given the form $e^{-x^2}$, or (iii) something else? $\endgroup$
    – user856
    Feb 27, 2018 at 4:16

1 Answer 1

1
$\begingroup$

I would say the notion of 'deriving' the normal distribution is a bit of a misnomer. There is no derivation, it simply is.

However the "derivation" you cite - where the Gaussian is a unique (up to a scale transformation and within reason) spherically symmetric distribution where the components are independent - was very important historically for motivating the Gaussian, both in the mathematical study of error and in physics (the Maxwell-Boltzmann distribution).

It's true that once we get down to one dimension, the distribution loses that motivation. Also, this is probably just my ignorance talking, but I know of no other way to compute the normalization factor without the trick of first squaring the distribution.

But there is another motivation that works for one dimension and is altogether more important, and that is the central limit theorem. This one goes back about as far as the first motivation, with DeMoivre's theorem that the distribution of the number of heads in a sequence of coinflips (i.e. a binomial distribution) looks more and more like a Gaussian as the number of coin flips gets large. Eventually it was discovered that this phenomenon is quite general. In fact, the sum of independent random variables from any distribution with finite variance becomes approximately normal as the number of variables becomes large ("becomes approximately normal" can be made mathematically precise.) The central limit theorem can be used to show that many quantities of statistical interest (maximum likelihood estimators, for instance) become approximately normal as the sample size gets large.

And it doesn't stop in one dimension, either. Independent vectors from multivariate normal distributions also obey a generalized central limit theorem. This motivates not only spherically-symmetric Gaussian distributions like the derivation you reference, but also ones that are asymmetric and even have correlation between the variables. Cause the sum of $n$ independent and identically distributed vectors whose mean vector and covariance matrix exist tend in distribution to a multivariate normal.

For reference, Jaynes' shall we say 'lively' textbook has a chapter with a lot of other historical derivation/motivation of the Gaussian. ('lively' = I wouldn't recommend this book generally for learning, especially to a beginner. It's very idiosyncratic and opinionated, though entertaining and interesting.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.