I assume $X\sim\mathcal{N}(\mu,\sigma)$ and wish to sample values but I am confused about different approaches and concepts that seem to be relevant for this problem.

It appears to me that this basically amounts to Inverse Transform Sampling, i.e., applying the quantile function of X to some uniformly distributed probability value from the domain of that quantile function (here, from $(0,1)$).

Since the quantile function of the normal distribution has no closed form solution according to wikipedia, this is not possible.

However, it is also stated here that for the quantile function of the normal distribution:

Thorough composite rational and polynomial approximations have been given by Wichura[6] and Acklam.[7] Non-composite rational approximations have been developed by Shaw.[8]

Incidently, in textbooks I also came across the Box-Muller method for sampling, which appears to me to be the single best practical approach also because those textbooks (while being rather brief on this topic) do not mention other approaches (but I am here to basically evaluate whether this is true and how to actually use Box-Muller). This method seems to attempt to sample directly (i.e., without attempting to construct the quantile function). However, it generates pairs of independent, standard, (normally) distributed random numbers.

  • Question Q1: Could one just use one of the resulting values of the Box-Muller method to obtain a sampling equivalently to applying the quantile function to some uniformly distributed probability value?
  • Question Q2: Is there some simple connection between the Box-Muller method and the quantile function? The seem so closely related but I have not seen a statement about it. So maybe I am completely off here.
  • Question Q3: Does it make sense to apply approximations such as those of Wishura, Aklam, and Shore mentioned above for sampling?
  • Question Q4: Am I correct that as a general approach for sampling arbitrary explicitly available CDFs, one could draw a uniformly distributed random probability from $[0,1]$ (or what makes sense for the CDF at hand) and then apply newtons method to obtain the sample approximately?
  • Question Q5: Am I corrrect that the further technique of Rejection Sampling could also be used for the normal distribution as it only requires the PDF, which is available in closed form? If so, is this just slower than, e.g., using Box-Muller?

1 Answer 1

  1. Since the Box-Muller method generates two IID standard normal realizations, discarding either one will result in a single standard normal realization, in as much as the marginal density is also standard normal. However, because we typically want to generate many realizations, it makes more sense to just take both, and if the sample size is odd, just discard the last observation.

  2. The connection is that the Box-Muller variates will have the standard bivariate normal distribution, so the quantile function for either marginal variable will be the same as the standard normal quantile. Proving the Box-Muller method in fact generates pairs of standard normal variables is a common exercise in undergraduate mathematical statistics courses.

  3. Approximating the quantile function (by whatever method) is not necessarily efficient for generating variates through the inverse transform method because it depends on the extent to which the approximation is faithful, and the computational cost to approximate the quantile.

  4. I don't understand this question. Please provide more mathematical details about what it is you are proposing.

  5. Rejection sampling is another possible approach. However, in general, such a sampling scheme requires a high chance of acceptance in order to be efficient. There are ways of achieving this, such as the Ziggurat algorithm.

There are other methods to efficiently sample from a standard normal distribution. See the Wikipedia article for more details.

  • $\begingroup$ For Q1+2: You refer with marginal variable and marginal density to the variables/densities of the two returned variates, right? For Q4: I saw this but, even simpler, approximating $x$ such that $F_X(x)$ equals a given (uniformly distributed) variate using bisection seems possible (albeit quite inefficient presumably). $\endgroup$ Sep 22, 2023 at 10:44
  • $\begingroup$ As an additional question: it appears as if there is no clear best (lets say reasonably accurate and efficient) sampling approach, right? For example, numpy has at least code for ziggurat and the R manual refers to Box-Muller. Are there important (possibly context-dependent) tradeoffs between these approaches that I am not aware of? $\endgroup$ Sep 22, 2023 at 10:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .