I am trying to understand why, in Monte Carlo importance sampling, the variance is zero when we choose a PDF that matches as closely as possible the function to integrate. The intuition is clear to me, but not the proof I am reading.

For an integral:

$$I = \displaystyle \int f(x) dx \tag{1} $$

the variance $\sigma^2$ is equal to:

$$ \displaystyle \sigma^2 = \frac{1}{N} \int (\frac{f(x)}{p(x)} - I)^2 p(x) dx \tag{2}$$

The approach is to find the minimum of the variance with the Lagrange multipliers method, considering as only boundary condition:

$$ \displaystyle \int p(x) dx = 1 \tag{3}$$

The function to minimize, as reported in the book, is:

$$\displaystyle L(p) = \int (\frac{f(x)}{p(x)})^2p(x) dx + \lambda \int p(x) dx \tag{4}$$

My first doubt is: why isn't the term $ -I$ in equation (2) considered in $L(p)$?

The partial derivative with respect to $p(x)$ of equation (4) gives:

$$p(x) = \frac{1}{\lambda}|f(x)| \tag{5}$$

My second doubt is: how can we say that, from equation (5):

$$ p(x) = \frac{|f(x)|}{\int f(x) dx} $$

Thank you!


1 Answer 1


Removing the $-I$ only changes the variance by a constant amount $-I^2/N$ (you can check the calculation simply since $I$ is constant with respect to $x$ and $\int p(x)dx=1$). So it doesn't affect the optimisation.

The values of $\lambda$ is just chosen to satisfy equation (3).

  • $\begingroup$ Thank you! Now it's clear. $\endgroup$
    – maurock
    Commented Oct 14, 2019 at 8:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .