How can one use a probability distribution to sample from a population

Question

Let us assume that we have a population and we interested in specific property of each element of this population. Let us assume further that this property follows a normal distribution X ~ P(M,Sigma). We can easily find the probability that a specific element has a certian property value , but how can we use this probability density to sample from this population ?

Answer 1

Here is an acceptance-rejection method that permits sampling from a density function $f(x)$, provided we can find a density function $g(x)$ from which we know how to sample and a constant $M$ such that $M*g(x) > f(x)$ for all $x$ in the support of $f(x).$

To be specific, suppose $f(x) = 1.1198 e^{-x^3},$ for $x > 0.$ One can show that this is a density function and that a random variable with this density has $E(X) = 0.506.$

Take $g(x) = 2\varphi(x),$ for $x > 0$, where $\varphi$ is the standard normal density. We say that $g(x)$ is the density of a folded (or half) normal distribution. (In R, the function rnorm samples from the normal distribution.) Furthermore, $3\varphi(x) = 1.5g(x) > f(x)$, for $x > 0.$

The program below, generates trial values from half normal, and accepts appropriate ones to give an independent random sample from $X$. We use the random sample to verify $E(X)$ and also to find $P(0.5 < X < 1.5),$ which cannot be found by ordinary analytical methods. (A histogram, shown below, illustrates that the sampled values closely approximate the target distribution.)

 m = 10^6;  t = abs(rnorm(m))          # m trial obs. from half normal
 acc.p = 1.1198*exp(-t^3)/(3*dnorm(t)) # m acceptance probabilities
 acc = rbinom(m, 1, acc.p)             # 1 = Acc, 0 = Rej
 x=t[acc==1]                           # accepted t's distributed as X 
 mean(x)                               # E(X)
 ## 0.5054726
 mean(x<1.5 & x>.5)                    # P(.5 < X < 1.5)
 ## 0.4526963

Histogram of about 667,000 accepted observations from $f(h)$, with graph of $f(h)$ (solid curve) and "envelope" $Mg(h)$ (dotted).

Notes: (1) About 2/3 of the trial points were accepted. A more tightly bounding envelope, if available, would give a higher acceptance rate. (2) The Metropolis-Hastings algorithm is a more complicated acceptance-rejection method used mainly for sampling from multivariate distributions. The simple univariate method illustrated here is similar in spirit to M-H, but (unlike M-H) it returns independent observations. (3) This method also illustrates importance sampling; the trial values are relatively concentrated in the region of the support where X's are most likely to occur.