3
$\begingroup$

Given a continuous multivariate pdf in analytical form (i.e. in function form), how can one sample from the corresponding distribution? In other words, what are the ways of coming up with random (or psuedo-random) realizations from the distribution with probability matching the one corresponding to the given pdf? What is the general idea/principle behind the sampling procedure?

Referring to related references would be helpful too.

$\endgroup$
1
  • 1
    $\begingroup$ Relevant reference: Luc Devroye 'Non-Uniform Random Variate Generation' $\endgroup$
    – zhoraster
    Jan 7, 2020 at 15:17

2 Answers 2

Reset to default
4
$\begingroup$

This is a broad topic. Here are a few examples to initiate some thinking on your part.

(1) To generate a random sample $x$ from a univariate distribution with CDF $F$, first generate a uniformly distributed random number $u \sim U(0,1)$ and take $x = F^{-1}(u)$. Note that $x$ has the desired distribution since

$$\mathbb{P}(x \leqslant a) = \mathbb{P}(F^{-1}(u) \leqslant a) = \mathbb{P}(u \leqslant F(a)) = F(a)$$

(2) To generate a random vector with a multivariate normal distibution, i.e., $\mathbf{x} \sim N(\mathbf{\mu}, \Sigma)$, first generate a vector $\mathbf{z}$ with components that are indpendent and distributed $z_j \sim N(0,1)$. Find the Cholesky decomposition of the covariance matrix $\Sigma = LL^T$ and take $\mathbf{x} = \mu + L\mathbf{z}$.

This imposes the desired covariance structure since

$$\mathbb{E}((\mathbf{x} - \mathbf{\mu})(\mathbf{x} - \mathbf{\mu})^T) = \mathbb{E}(L\mathbf{z}(L\mathbf{z})^T)= \mathbb{E}(L\mathbf{z}\mathbf{z}^T L^T) = L \mathbb{E}(\mathbf{z}\mathbf{z}^T)L^T = LL^T = \Sigma$$

(3) More generally, consider an approach like Gibbs sampling.

Also you could explore the notion of a copula if the marginal distributions are accesible.

$\endgroup$
4
  • $\begingroup$ Awesome examples. Could you please elaborate more about sampling with copula? If I already have a sample from the joint copula, sampling from the joint distribution would be easy, otherwise it seems that sampling from a copula is similarly hard as sampling from the general distribution? $\endgroup$
    – user25004
    Jan 7, 2020 at 1:58
  • $\begingroup$ @user25004: I’ll get back to you on that when I get a chance. $\endgroup$
    – RRL
    Jan 7, 2020 at 3:55
  • $\begingroup$ Great. Thanks! Looking forward to it. $\endgroup$
    – user25004
    Jan 7, 2020 at 14:31
  • 1
    $\begingroup$ @user25004: I don't have a comprehensive answer. The topic appears in a number of questions posted on this and other SE sites, for example here. I am familiar with some copula-specific methods (as per point (2) in the linked answer). As an example, we can sample from a t-distribution or copula by first generating a random vector $\mathbf{Z} \sim N(0, \Sigma)$ and a chi-squared distributed random number $s \sim \sqrt{\chi^2(\nu)/\nu}$ and then taking $\mathbf{X} = s^{-1}\mathbf{Z}$. In this case we have $\mathbf{X} \sim t_\nu(0, \Sigma)$. $\endgroup$
    – RRL
    Jan 7, 2020 at 19:15
0
$\begingroup$

Generate a random number $x$ and output the smallest $y$ such that $F(y) \ge x$ where $F$ is the cdf of your distribution? What are you looking for exactly?

$\endgroup$
1
  • $\begingroup$ The question is about high dimensional random variables. $\endgroup$
    – user25004
    Jan 7, 2020 at 1:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .