I recently asked this question (link here) regarding generating samples of a Gaussian vector with independent components above a hyperplane. Before getting an answer, I had already performed a simple simulation in 2-D: Let $Z=(Z_1,Z_2)$ be a Gaussian vector with $N(0,1)$ independent components. I tried generating samples of $Z$ such that $Z_1>4$ (i.e. my hyperplane) using what I thought was an intuitive approach:

1) Generate a sample of $Z_2 \sim N(0,1)$.

2) Since $Z_1,Z_2$ are independent, I just had to generate a sample of $Z_1 > 4$.

3) Then this gives me a sample of $Z = (Z_1,Z_2)$.

Now, I am interested in generating samples of a Gaussian vector that lie above a hypersurface. For simplicity, let's consider the 2-D example in which I want to generate $Z$ (same vector above) such that $Z_1 > Z_2^2 + 3$. I adapted the "intuitive" approach I pursued above as follows:

1) Generate a sample of $Z_2 \sim N(0,1)$ which we denote by $z_2$.$

2) Then generate a sample of $Z_1$ that satisfies $Z_1 > z_2^2 + 3$.

I compared the samples I obtained from the above approach with what I would get from standard rejection sampling. However, I realized that the above approach is wrong.

I am trying to understand why the method I described above fails for the hypersurface I tried, but I don't see it intuitively. Since the above approach fails for the hypersurface I tried, is there a general efficient approach to generate samples of a Gaussian vector with independent components above a hypersurface assuming that the functional form of this hypersurface is known? Note that I'm ruling out standard rejection sampling because it is highly inefficient for rare events.

Thoughts on either question (especially why the above approach fails) greatly appreciated.

  • $\begingroup$ It fails because the distribution of $Z_2$ is no longer normal. For example, consider the condition $z_1\ge 5\,{\rm sgn}(z_2)$. There is very little probability that $Z_2>0$. $\endgroup$ – Rahul Jul 23 '18 at 2:34
  • $\begingroup$ How come the method does not fail when the hypersurface is a hyperplane? $\endgroup$ – Tomas Jorovic Jul 23 '18 at 14:32
  • $\begingroup$ Because $Z_1$ and $Z_2$ are independent, so if you impose a condition only on $Z_1$ it does not affect the marginal distribution of $Z_2$. But if you impose a condition involving both $Z_1$ and $Z_2$, then they cannot remain independent, and the marginal distribution of $Z_2$ may change. $\endgroup$ – Rahul Jul 24 '18 at 6:05

Perhaps what's wrong with your method is best illustrated by a related method that works.

You can generate $Z_2$ as before, generate $Z_1\gt3$ as in the other question, and then accept or reject the entire sample (including $z_2$) according as $Z_1\gt z_2^2+3$. This is a more efficient “semi-rejection” sampling that accepts about $36\%$ of the samples, compared to $0.05\%$ in standard rejection sampling.

This method produces a different distribution than yours, since larger values of $z_2$ get rejected more often. When the surface is a plane parallel to the $z_2$ direction, all lines parallel to the $z_2$ direction are entirely on one side of the plane, so the plane doesn't change the distribution along $z_2$. By contrast, when the surface is curved, as in the case of $Z_1\gt Z_2^2+3$, then parts of the lines parallel to the $z_2$ direction are cut off, and thus $Z_2$ is no longer normally distributed.

  • $\begingroup$ Thanks for the response again, Joriki. I have yet to review the response you gave in the previous question. In your answer above, how can I justify generating $Z_1 > 3$? This sounds arbitrary. So can I just generate any $Z_1 > k$ then and check if $Z_1 > z_2^2 + 3$? $\endgroup$ – Tomas Jorovic Jul 23 '18 at 14:31
  • $\begingroup$ @TomasJorovic: It's not arbitary. It works because $Z_1$ and $Z_2$ are independent, so any conditions you impose on $Z_1$ don't change the distribution of $Z_2$. But that's only true if they're really only conditions on $Z_1$, not something like $Z_1\gt Z_2^2+3$, which correlates $Z_1$ and $Z_2$. (See also the last paragraph in the answer.) Yes, you can do that with any $k\le3$, but of course it will be less efficient for smaller $k$. You can't do it with $k\gt3$ because then you wouldn't be covering the entire area $Z_1\gt Z_2^2+3$. $\endgroup$ – joriki Jul 23 '18 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.