# Sampling from a distribution with given pdf

Given a continuous multivariate pdf in analytical form (i.e. in function form), how can one sample from the corresponding distribution? In other words, what are the ways of coming up with random (or psuedo-random) realizations from the distribution with probability matching the one corresponding to the given pdf? What is the general idea/principle behind the sampling procedure?

Referring to related references would be helpful too.

• Relevant reference: Luc Devroye 'Non-Uniform Random Variate Generation' Jan 7, 2020 at 15:17

This is a broad topic. Here are a few examples to initiate some thinking on your part.

(1) To generate a random sample $$x$$ from a univariate distribution with CDF $$F$$, first generate a uniformly distributed random number $$u \sim U(0,1)$$ and take $$x = F^{-1}(u)$$. Note that $$x$$ has the desired distribution since

$$\mathbb{P}(x \leqslant a) = \mathbb{P}(F^{-1}(u) \leqslant a) = \mathbb{P}(u \leqslant F(a)) = F(a)$$

(2) To generate a random vector with a multivariate normal distibution, i.e., $$\mathbf{x} \sim N(\mathbf{\mu}, \Sigma)$$, first generate a vector $$\mathbf{z}$$ with components that are indpendent and distributed $$z_j \sim N(0,1)$$. Find the Cholesky decomposition of the covariance matrix $$\Sigma = LL^T$$ and take $$\mathbf{x} = \mu + L\mathbf{z}$$.

This imposes the desired covariance structure since

$$\mathbb{E}((\mathbf{x} - \mathbf{\mu})(\mathbf{x} - \mathbf{\mu})^T) = \mathbb{E}(L\mathbf{z}(L\mathbf{z})^T)= \mathbb{E}(L\mathbf{z}\mathbf{z}^T L^T) = L \mathbb{E}(\mathbf{z}\mathbf{z}^T)L^T = LL^T = \Sigma$$

(3) More generally, consider an approach like Gibbs sampling.

Also you could explore the notion of a copula if the marginal distributions are accesible.

• Awesome examples. Could you please elaborate more about sampling with copula? If I already have a sample from the joint copula, sampling from the joint distribution would be easy, otherwise it seems that sampling from a copula is similarly hard as sampling from the general distribution? Jan 7, 2020 at 1:58
• @user25004: I’ll get back to you on that when I get a chance.
– RRL
Jan 7, 2020 at 3:55
• Great. Thanks! Looking forward to it. Jan 7, 2020 at 14:31
• @user25004: I don't have a comprehensive answer. The topic appears in a number of questions posted on this and other SE sites, for example here. I am familiar with some copula-specific methods (as per point (2) in the linked answer). As an example, we can sample from a t-distribution or copula by first generating a random vector $\mathbf{Z} \sim N(0, \Sigma)$ and a chi-squared distributed random number $s \sim \sqrt{\chi^2(\nu)/\nu}$ and then taking $\mathbf{X} = s^{-1}\mathbf{Z}$. In this case we have $\mathbf{X} \sim t_\nu(0, \Sigma)$.
– RRL
Jan 7, 2020 at 19:15

Generate a random number $$x$$ and output the smallest $$y$$ such that $$F(y) \ge x$$ where $$F$$ is the cdf of your distribution? What are you looking for exactly?

• The question is about high dimensional random variables. Jan 7, 2020 at 1:14