Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

For a Monte-Carlo integration of

$$\int_\Omega P(x)f(x)\ \text d x,$$

there seems to be no apriori distinction if $f$ or $P$ is the probability function. So does it matter if I consider

$$P, f, P f, \text{ or 1 over all of } \Omega$$

to be the distribution function I draw the random points $x$ from, given that I use the complement function as object to plug these values in?

share|improve this question
1  
First, I suspect you mean density function not distribution function. Second, it is not likely that all of $P$, $f$ $Pf$ and $1$ are density functions (without scaling). But if you are willing to scale, then no. In fact, one often uses a density $g$ and computes $$\int_\Omega\frac{P(x)f(x)}{g(x)}g(x)dx$$. This is called importance sampling. –  deinst Mar 15 '12 at 12:41
    
@deinst: Oh yes, my bad, you're right. I didn't mention any normalization properties. –  NikolajK Mar 15 '12 at 16:52

2 Answers 2

up vote 1 down vote accepted

The short answer is no, it does not matter which probability density function you use.

There are two properties that a density function $f$ must have:

  1. The density function $f$ must be greater than 0 everywhere on $\Omega$.
  2. The integral $\int_\Omega f(x) dx$ must equal 1.

If $P(x)$, $P(x)f(x)$ or 1 satisfy both conditions, then yes they can be used for the density of the sampling distribution.

If a function $g(x)$ satisfies just the first condition, and has a finite integral over $\Omega$, then there is some normalizing coefficient $c$ that will make $$\int_\Omega \frac{g(x)}{c}dx=1$$

This is particularly interesting if you choose $g(x)=P(x)f(x)$ then your integral becomes $$\int_\Omega c \frac{g(x)}{c} dx$$ drawing samples $x_i$ from a probability distribution with density $g$, and evaluing the constant $c$ at each $x_i$ you get a Monte Carlo estimate with variance 0! Which would gbe a good thing except that your normalizing constant $c$ is just the integral you started with.

In the technique called Importance Sampling one deliberately chooses a density $g$ different from $f$ and computes the integral $$\int_\Omega \frac{P(x)f(x)}{g(x)}g(x)dx$$ by sampling from $g$. With a well chosen $g$ one can often get a much better estimate than straight a Monte Carlo estimation.

share|improve this answer

As long as both functions are integrable, the distinction technically doesn't matter because you could use methods like Metropolis-Hastings that are not sensitive to the normalizing constant.

However, this is a misleading thing to do. Typically, you've got some function, let's say it is $f(x)$, and you want to compute a probability integral of $f(x)$ to get its expectation under some probability distribution $P(x)$ which has a density function of $p(x)$. In the case where you know what you want the expectation of, it would be harmful to represent $f$ as the distribution and $P$ as the thing-you're-taking-expectation-of.

Then, really you're secretly doing a Stieltjes integral of the form $$E[f]_{P} = \int f(x)d[P(x)].$$ As a matter of convenience for almost any distribution of practical interest, this simplifies to $$E[f]_{P} = \int f(x)p(x)dx.$$

And so, if you can draw the $x$ values in such a way that you can guarantee that they follow the distribution $P(x)$, then you can approximate this integral with the sample mean: $$ E[f]_{P} \approx \frac{1}{N}\sum_{i=1}^{N} f(x_{i}),$$ adjusted for different domains of integration, of course.

Added: To reflect the comments.

It won't be mathematically incorrect if you just use merely integrable $P(x)$ and $f(x)$. You can conceptually think of Monte Carlo as a generic procedure for computing the Stieltjes integral of some function with respect to some other function, as long as the function that appears in the differential satisfies the monotonicity type conditions needed for Stieltjes integrals.

The reason why this may not be a great way to present it is (a) in practice this will almost always reduce to a probability calculation, because the random sampling procedures will draw $x_{i}$ values in accordance with probability distributions that are proportional to the Stieltjes function you care about, and thus you'll have to know that function's normalizing constant in order to correct for this if you really wanted the Stieltjes integral and not the probability integral; and (b) Any Stieltjes integral that you're attempting to solve numerically must inherently reduce down to a Riemann integral anyway, and so why bother making the distinction between Monte Carlo for general Riemann integrals vs. Monte Carlo for Stieltjes integrals?

share|improve this answer
    
Thanks for the answer. I don't see why it would be harmful yet. –  NikolajK Mar 15 '12 at 16:49
    
Well to put it another way, at some point you have to draw values for the samples $x_{i}$. How are you going to do that? Will it be with a built-in random number generator and functions that sample from a known probability distribution? That's sampling from $P(x)$ and then evaluating $f(x_{i})$. What if your probability function $P(x)$ is too complicated to use built-in methods? Well, you still have to decide on how you'll generate your $x$ values, and that definitely matters regarding what you ultimately get out of the problem. –  EMS Mar 15 '12 at 17:00
    
Another thing to consider: If you view the "probability" function as being $f(x)$ (which is perhaps some unrelated function and is not normalized and was never considered as a probability function until you decided to think of it that way), then you're sort of flipping Monte Carlo around and taking the weighted expectation of a genuine probability function $P(x_{i})$ w.r.t. weights that won't be between 0 and 1 coming from some other thing $f(x)$. Nevermind the issue of the scale factor... why would you want to consider it a weighted expectation of a probability distribution? –  EMS Mar 15 '12 at 17:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.