0
$\begingroup$

I'm studying probability. I came a cross "sampling from distributions". Given a probability density function $f_X(x)$, what I understood is that sampling means getting values of $x$ according to the distribution of probabilities over the $x$s. So that a value of $x$ (or an interval of $x$) that has a probability mass or density more than others will have a higher chance to appear in the sampling than others.

I have three questions here:

1- is my previous understanding correct?

2- I often see people sample from a normal distribution with zero mean and variance of 1. Why is that?

3- Does sampling from a normal distribution with zero mean and variance of 1 means that the chance of getting zeros or numbers around zero is higher than other numbers?

$\endgroup$
1
$\begingroup$

Yes your understanding is correct, although for continuous distributions every exact number has probability $0$, which is almost a paradox, but people get around this by talking about the probability of a sample being in a set like an interval, which has positive probability. The normal distribution $N(0,1)$ can be used to generate samples from any Gaussian distribution $N(\mu, \sigma)$ and even higher dimensional Gaussian distributions, so it's sort of a "building block" distribution for sampling from other distributions. The normal distribution is monotonically decreasing as you move away from zero, so yes, numbers closer to zero are more likely than numbers farther away from zero.

$\endgroup$
  • $\begingroup$ but I don't get it, isn't the Normal distribution is just a Gaussian distribution?! Is there any difference? $\endgroup$ – Jack Twain Apr 5 '14 at 19:18
  • $\begingroup$ also why people occasionally sample from a standard normal distribution? I think you missed this part in your answer. Almost always I see that people add noise to values by sampling from a standard normal distribution. Why not other distributions? Why even a standard normal distribtuion, i.e. why a mean of 0 and variance of 1 not other values? $\endgroup$ – Jack Twain Apr 5 '14 at 19:21
  • $\begingroup$ A Gaussian distribution can have mean and variance different from $0,1$. Normal means mean = 0 and variance = 1. Also Guassian distributions can be defined in higher dimensions in terms of a covariance matrix. I'm not sure why people would use samples directly from the normal distribution $N(0,1)$. The mean and the variance of the Gaussian distribution are important when you want to use the samples in practice, and there's nothing really special about having a variance of $1$. Having a mean of $0$ can be special and used, e.g. if you assume errors have mean $0$ so the errors are unbiased. $\endgroup$ – user2566092 Apr 6 '14 at 17:58
0
$\begingroup$

The Normal distribution is exactly the same thing as a Gaussian distribution, there are just two names.

People use a lot the normal distribution because of the Central Limit Theorem : it states that the sample mean of an independent and identically distributed sample converges in distribution to a normal distribution. This is true whatever the original distribution of the random sample.

Thus the normal distribution is usually appropriated to model the noise. If the sample is big enough, there is a good chance that the noise will follow a normal distribution. Furthermore, the normal distribution is symmetric (e.g. you wouldn't use a Gamma or Chi square distribution for the noise, since they only take positive values).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.