3
$\begingroup$

I'm currently studying about sampling methods, and some terminology that has arisen is that a distribution is "hard to sample from".

What does this mean?

My current intuition is that the distribution is joint and we need to collect a lot of values to get a sample, or that for example the variables of the distribution are in a very high dimension, and you need to sample A LOT of data to understand how it looks like. Is this correct?

$\endgroup$
4
  • $\begingroup$ My understanding is that it means there is no efficient procedure for computing a random variable that has the given distribution. $\endgroup$
    – user856
    Jun 6, 2019 at 12:44
  • $\begingroup$ Related: Why is it hard to directly sample from certain statistical distributions? $\endgroup$
    – user856
    Jun 6, 2019 at 12:49
  • $\begingroup$ @Rahul What do you mean by "computing a random variable"? $\endgroup$ Jun 6, 2019 at 22:39
  • $\begingroup$ Sorry, I just meant computing a sample. $\endgroup$
    – user856
    Jun 7, 2019 at 6:09

1 Answer 1

2
$\begingroup$

Good question. Very foundational to learning the true value of efficient sampling techniques.

It boils down to the simple fact that to "sample from a distribution" you need to know the CDF of the distribution. More precisely you need the Inverse CDF i.e. a function that will give you the a sample if you feed it a probability. Although all CDFs are monotonically increasing and hence theoretically invertible analytically (practically) it not that easy.

So two issues prevent you from directly getting values from the CDF. One is getting the CDF itself and the second is inverting it. The problem gets more complex in higher dimensions. So efficient sampling techniques are used to approximate the functions instead.

Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.