1
$\begingroup$

My question is a little complicated, so I'm going to set up some background first.

Let's imagine that I'm taking samples from a Bernoulli distribution, and I need to choose a value for $p$ (the chance of finding a positive result). Assume that, for whatever reason, I want to integrate the probability of encountering one positive result, followed by one negative result, over all possible values of $p$. This is pretty simple. We'd just compute:

$\int_{0}^{1}p(1-p)dp$

Now let's say that there are three possible outcomes, rather than two. Let's call these outcomes $1$, $2$, and $3$, and denote the probability of finding a $1$ with $p$, the probability of finding a $2$ with $q$, and the probability of finding a $3$ with $(1-p-q)$. Once again, if I wanted to integrate over the probability of finding a $1$, then a $2$, then a $3$ for all possible parameterizations of the pmf assigned to this sample space, I would compute the integral:

$\int_{0}^{1}(\int_{0}^{1-p}pq(1-p-q)dq)dp$

Adding a fourth possibility (call it $4$, and add a new parameter $r$), the integral becomes

$\int_{0}^{1}(\int_{0}^{1-p}(\int_{0}^{1-p-q}pqr(1-p-q-r)dr)dq)dp$

... and so on. This can be extended for however many discrete events you choose.

What I want to do is find a continuous analog for this kind of integration over parameters, where instead of integrating over individual values for $p$, $q$, etc., I integrate over the space of all possible probability density functions.

To give a crude example, suppose I took 5 samples from the real numbers between $0$ and $1$ (inclusive) with an unknown probability density function, and find the values $0$, $0.401$, $0.72$, $0.27651$, and $0.8$. The uniform distribution on $[0, 1]$ gives me a likelihood function of the product of the probability density of the uniform distribution at these 5 values. However, there are infinitely many pdfs which could have generated these 5 values; I want to integrate over the likelihood function of every single one of them.

I have no idea if this is even possible. My intuition is to treat this continuous analog as a limiting case of the discrete version, i.e. integrating over $n - 1$ parameters for $n$ potential discrete events, as $n$ approaches infinity. However, given that this would involve evaluating infinitely many integrals (which I cannot do, given that my lifespan is finite), I'm assuming there must be a better approach.

Does such a continuous analog exist, and if so, how does one evaluate it?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.