1
$\begingroup$

As I understood it, the 'normal distribution' is $$\frac{1}{\sqrt{2\pi}}\exp\left(\frac{-(x-\mu)^2}{2{\sigma}^2}\right)$$ Now according to this the 'normal probability density function' is $$f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp\left(\frac{-(x-\mu)^2}{2{\sigma}^2}\right)$$ and according to this at the top of page 466 (or 436 if you own the book) the 'normalised Gaussian distribution' is $$f(t)=\frac{1}{\tau \sqrt{2\pi}}\exp\left(\frac{-t^2}{2{\tau}^2}\right)$$

In the cases above, $\mu =$ mean, $\sigma =$ standard deviation and $\tau =\Delta t$

I am very confused; each time I search the web for answers I get different words and formulae, this post is designed to try to dispel this confusion.

Could someone please explain the meaning and difference between the formulae given above and the similarity and/or differences between the phrases used in the title and body?

Thank you.

$\endgroup$
7
  • 1
    $\begingroup$ Compare $$\frac{1}{\sqrt{2\pi \sigma^2}}\mathrm{e}^{-\frac{(x-\mu)^2}{2\sigma^2}} = N(\mu,\sigma^2)$$ with $\mu = 0$? this is the guassian centered about mean of zero. This occurs when you shift the (first one you wrote) distribution to be zero mean fluctuations. $\endgroup$
    – Chinny84
    Sep 29, 2015 at 15:18
  • 1
    $\begingroup$ That would be the full blown standard normal distribution. But in the example you gave above it is $\sigma = \tau = \Delta t$ so not quite. Though you could rescale the distribution. $\endgroup$
    – Chinny84
    Sep 29, 2015 at 15:21
  • 1
    $\begingroup$ I think that Gaussian and Normal distribution are two different names for the same thing. $\endgroup$
    – Zoran Loncarevic
    Sep 29, 2015 at 15:25
  • 1
    $\begingroup$ Nothing is different between the two. Though "Normal" is often the more used phrase in stats/probability. I am not sure why that is the case..Though Gauss was the first to use the distribution in terms of measurement errors.for orbits I think. $\endgroup$
    – Chinny84
    Sep 29, 2015 at 15:26
  • 1
    $\begingroup$ de Moivre used this before Gauss, but not for measurement errors. I think Guass showed that least-squares estimates coincide with maximum-likelihood estimates only if the distribution of the errors is normal. $\endgroup$
    – Michael Hardy
    Oct 2, 2015 at 17:48

3 Answers 3

Reset to default
2
$\begingroup$

The second formula is the standard expression for the probability density function (PDF) corresponding to the normal (or Gaussian) distribution with mean $\mu$ and standard deviation $\sigma$. As it is a PDF, it is normalised to 1, i.e., its integral over admissible values of $x$ is $1$. The first formula is missing the $1/\sigma$ factor, thus it is not a PDF. Finally, the third formula can be obtained from the second one with direct substitution $\sigma\rightarrow\tau$, $x\rightarrow t$, and $\mu\rightarrow0$.

$\endgroup$
3
  • $\begingroup$ Thanks for your reply, I thought the first formula was a cdf? $\endgroup$
    – BLAZE
    Sep 29, 2015 at 15:36
  • 1
    $\begingroup$ CDF is the cumulative function, i.e., in loose terms, CDF(x) can be interpreted as the probability that a number randomly generated from the corresponding distribution is less than or equal to x - see the table at en.wikipedia.org/wiki/Normal_distribution $\endgroup$
    – ewcz
    Sep 29, 2015 at 15:40
  • $\begingroup$ According to the text I have in front of me the first formula is correct. Yourself and stochasticboy321 have said that it is missing a factor $\cfrac{1}{\sigma}$, but does the formula have any meaning without the $\cfrac{1}{\sigma}$ factor? $\endgroup$
    – BLAZE
    Sep 29, 2015 at 15:45
2
$\begingroup$

A gaussian distribution is the same as a normal distribution. The standard gaussian or standard normal distribution is the gaussian distribution with $\mu = 0$, $\sigma = 1$.

BTW, the first equation above is incorrect, as its integral over the reals is $\sigma$, and not 1. As the probability of the sample space, by definition, has to be 1, any distribution must integrate to, well, 1.

You could normalise this by dividing it by $\sigma$ (to get the second equation). Note that this normalisation has nothing to do with the normal in normal distribution. The distribution's name comes from it being normal in the sense of usual, and the process of making the probability of $\mathbb{R}$ (or whatever your sample space it) equal 1 comes from making it fit to the norm, or rule.

$\endgroup$
10
  • 1
    $\begingroup$ Perhaps it'll be better to leave that up? That motivated the spiel about normalising a distribution, which is useful for someone else who is unfamiliar with the subject and might come across this later. $\endgroup$
    – stochasticboy321
    Sep 29, 2015 at 15:37
  • 1
    $\begingroup$ A fundamental fact about a distribution is that it sums/integrates to 1. This is the 2$^\text{nd}$ of Kolmogorov's axioms. It makes tenuous sense at best to call the expression a distribution without the $1/\sigma$ factor. To the extent of my know-how, if you call it a distribution, it forces $\sigma = 1$. $\endgroup$
    – stochasticboy321
    Sep 29, 2015 at 15:51
  • 1
    $\begingroup$ Yes! It looks like a bell curve, just doesn't add up. (In case this seems pedantic, a whole bunch of physics finds great use for, and is held up mainly by the difficulty in computing, the normalising factor for various distributions. Check out some statistical mechanics sometime :) ) $\endgroup$
    – stochasticboy321
    Sep 29, 2015 at 16:17
  • 1
    $\begingroup$ Well, take any non-negative function over any set such that it is $L_1$ integrable. This is currently not a probability distribution. But, if you divided it by the integral, it will become. So, $\int_\mathbb{R} f(x) ~dx$ is called the normalising factor. In physical contexts, one posits that a particle in a certain potential has a probability proportional to some function of position, temperature, etc. of having some energy $E$, through some physical justification. The normalising factor then exhibits most of the macroscopic behaviour ($PV = nRT$ ,$C_V$, etc.) that you'll come across. $\endgroup$
    – stochasticboy321
    Sep 29, 2015 at 17:19
  • 1
    $\begingroup$ Consequently, you'd like to know it for whatever system you're interested in. It turns out that this is pretty expensive to calculate, which is a pretty major bane. More would lead to a textbook, but let's suffice it to say that the normalisation is important mathematically for probabilities to make sense, and has non-zero importance in non-mathematical contexts as well. $\endgroup$
    – stochasticboy321
    Sep 29, 2015 at 17:21
1
$\begingroup$

The normal distibution, also called the Gaussian distribution is the probability distribution that assigns to every measurable set $A$ of real numbers the probability $$ \int_A \frac 1 {\sqrt{2\pi}} \exp\left( \frac{-1} 2 \left( \frac{x-\mu} \sigma \right)^2 \right) \, \frac{dx} \sigma. $$ In particular, there is the cumulative probability distribution function of the normal distribution: $$ x\mapsto \int_{-\infty}^x {\sqrt{2\pi}} \exp\left( \frac{-1} 2 \left( \frac{w-\mu} \sigma \right)^2 \right) \, \frac{dw} \sigma. $$

The normal probability density function, also called the Gaussian probability density function, is $$ x\mapsto \frac 1 {\sqrt{2\pi}} \exp\left( \frac{-1} 2 \left( \frac{x-\mu} \sigma \right)^2 \right) \cdot \frac 1 \sigma. $$ If you have any trouble remembering which is which, then remember the meanings of the terms probability distribution and probability density function. As with all density function, whether we're talking about probability density or mass density or energy density or population density, the value of the density is an intensive quantity (density) and its integral is an extensive quantity (probability or mass or energy or population).

I suspect the word "normalised" is intended to indicate that it has a standard deviation depending on a parameter. That is $\tau$. But that has nothing to do with the word "normal" as used above. The word "normal" is somewhat overburdened.

"Gaussian" is something of a misnomer as well. Abraham de Moivre identified the importance of this particular probability distribution in the first half of the 18th century, well before Carl Gauss was born (Gauss was born in the 1770s).

$\endgroup$
6
  • $\begingroup$ Thank you for your rigorous answer. I have one question though, I'm not clear on the difference between probability distribution and probability density function. Could you elaborate? I tried looking at this related thread but still unsure. $\endgroup$
    – BLAZE
    Oct 2, 2015 at 15:28
  • $\begingroup$ Also would you mind taking a look at this and tell me what I'm doing wrong please? $\endgroup$
    – BLAZE
    Oct 2, 2015 at 15:38
  • 1
    $\begingroup$ @BLAZE : A probability distribution assigns a probability to every measuarble set: $A \mapsto \Pr(A)$. The input is a set $A$ and the output is the probability of that set. A probability density function is a function that gets integrated over a set to get the probability of the set: thus $x\mapsto f(x)$ is a probability density if $\displaystyle A\mapsto \int_A f(x)\,dx$ is the corresponding probability distribution. A probability distribution on measurable sets of real numbers is determined by its cumulative distribution function $x\mapsto\Pr\Big(\,(-\infty,x]\,\Big)$. ${}\qquad{}$ $\endgroup$
    – Michael Hardy
    Oct 2, 2015 at 17:38
  • 1
    $\begingroup$ That's perfect, exactly what I needed to know, I cannot thank you enough, regards. $\endgroup$
    – BLAZE
    Oct 2, 2015 at 17:41
  • 1
    $\begingroup$ When I say "is determined by", that means two different probability distributions cannot have the same cumulative probability distribution function. If they both assign the same probability to every set of the form $(-\infty,x]$, then then both assign the same probability to EVERY measurable set. ${}\qquad{}$ $\endgroup$
    – Michael Hardy
    Oct 2, 2015 at 17:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .