Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am studying normal distribution for the first time and I'm having trouble understanding where this formula came from:

$$\phi(x) = \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} x^2}$$

Could someone derive this equation?

How about in the general case?

$$\frac{1}{\sigma\sqrt{2\pi}} e^{ -\frac{(x-\mu)^2}{2\sigma^2} }$$

share|improve this question

3 Answers 3

up vote 2 down vote accepted

Here's a commonly seen derivation.

Consider throwing a dart at a dartboard, aiming at the origin. We make the assumptions that:

  1. The errors in the horizontal and vertical directions are independent and identically distributed
  2. Errors are isotropic, i.e. the magnitude of the error doesn't depend on direction
  3. The chance of the dart landing in a small area is proportional to the area
  4. Large errors are less likely than small errors

Say that the probability of landing in a strip centered on $x$ and of width $\Delta x$ is $p(x)\Delta x$, and similarly $p(y)\Delta y$ for landing in a strip centered at $y$ of width $\Delta y$.

Since horizontal and vertical errors are independent we can multiply these probabilities to get the probability of landing in a box at $(x,y)$ of size $\Delta x\Delta y$. By assumptions (1) and (2) this should only depend on the distance from the origin, and should also be proportional to $\Delta x\Delta y$:

$$p(x)\Delta x \cdot p(y)\Delta y = f(r) \Delta x \Delta y$$

which tells us that

$$p(x)p(y) = f(r)$$

If we differentiate with respect to the angular coordinate $\theta$ on both sides, we get

$$p(x) \frac{\partial p(y)}{\partial \theta} + p(y) \frac{\partial p(x)}{\partial \theta} = 0$$

which, using polar coordinates $x=r\cos\theta$ and $y=r\sin\theta$ becomes

$$p(x)p'(y) r\cos\theta - p(y)p'(x) r\sin\theta = 0$$

or

$$p(x)p'(y)x - p(y)p'(x) y = 0$$

which can be rearranged to

$$\frac{p(x)x}{p'(x)} = \frac{p(y)y}{p'(y)}$$

Since both sides are functions of an independent variable and yet are equal, they must be equal to a constant:

$$xp(x) = Cp'(x)$$

and we can now separate variables and integrate, getting

$$\frac{x^2}{2} = C\log p(x) + b$$

or

$$p(x) = A \exp\left(\frac{x^2}{2C} \right)$$

Now the assumption that large errors are less likely than small tells us that $C<0$, and the constant $A$ is determined by the requirement that the probability integrates to 1.

share|improve this answer

Here's another derivation. Consider a random variable $X$ with distribution $p(x)$, which has zero expectation and unit mean: $E(X)=0$ and $E(X^2)=1$.

Following the principle of maximum entropy we try to find the distribution with maximal entropy, subject to the conditions on the mean and variance. This gives a Lagrangian optimization problem, with Lagrangian

$$L(p) = \int p(x) \log p(x) dx - a\left( \int p(x)dx - 1\right) - b\int xp(x)dx - c\left( \int x^2p(x) dx - 1\right)$$

Differentiating once,

$$\delta L(p) = \int (\delta p \log p + 1)dx - a\int\delta p dx - b\int x\delta p dx - c\int x^2\delta p dx$$

and requiring that this be zero gives

$$\delta L(p) = \int \left(\log p + 1 - a - bx - cx^2\right) \delta p dx = 0$$

which holds for all variations $\delta p$ if and only if

$$\log p(x) = a-1 + bx + cx^2$$

or

$$p(x) = A e^{bx + cx^2}$$

The constants $a$, $b$ and $c$ are determined by requiring that $E(X)=0$, $E(X^2)=1$ and that $p(x)$ integrates to 1.

share|improve this answer

From wikipedia: Looks like De Moivre was almost there but didn't realize it circa 1768, then 71 years later, Gauss got there. De Moivre started to discover what we now know, that the normal distribution is a limiting case of the binomial distribution with $p=\frac12$ as $n\rightarrow\infty$. (I say started, because he lacked the concept of a probability density function. He had the crucial formula, but not the concept to develop it into a distribution in its own right.) This is one way to define the normal distribution. Gauss, while developing the method of maximum likelihood estimation, showed that the normal curve is the only distribution which justifies the arithmetic mean of a set of measurements, well-known at the time, as the choice for estimating the location parameter.

The general case follows from properties of integrals and/or expected value, variance, and transformation of random variables, analogously to shifts $f(x-a)$ and dilations $f(cx)$ of an arbitrary function $f(x)$.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.