My question is a little complicated, so I'm going to set up some background first.

Let's imagine that I'm taking samples from a Bernoulli distribution, and I need to choose a value for $p$ (the chance of finding a positive result). Assume that, for whatever reason, I want to integrate the probability of encountering one positive result, followed by one negative result, over all possible values of $p$. This is pretty simple. We'd just compute:


Now let's say that there are three possible outcomes, rather than two. Let's call these outcomes $1$, $2$, and $3$, and denote the probability of finding a $1$ with $p$, the probability of finding a $2$ with $q$, and the probability of finding a $3$ with $(1-p-q)$. Once again, if I wanted to integrate over the probability of finding a $1$, then a $2$, then a $3$ for all possible parameterizations of the pmf assigned to this sample space, I would compute the integral:


Adding a fourth possibility (call it $4$, and add a new parameter $r$), the integral becomes


... and so on. This can be extended for however many discrete events you choose.

What I want to do is find a continuous analog for this kind of integration over parameters, where instead of integrating over individual values for $p$, $q$, etc., I integrate over the space of all possible probability density functions.

To give a crude example, suppose I took 5 samples from the real numbers between $0$ and $1$ (inclusive) with an unknown probability density function, and find the values $0$, $0.401$, $0.72$, $0.27651$, and $0.8$. The uniform distribution on $[0, 1]$ gives me a likelihood function of the product of the probability density of the uniform distribution at these 5 values. However, there are infinitely many pdfs which could have generated these 5 values; I want to integrate over the likelihood function of every single one of them.

I have no idea if this is even possible. My intuition is to treat this continuous analog as a limiting case of the discrete version, i.e. integrating over $n - 1$ parameters for $n$ potential discrete events, as $n$ approaches infinity. However, given that this would involve evaluating infinitely many integrals (which I cannot do, given that my lifespan is finite), I'm assuming there must be a better approach.

Does such a continuous analog exist, and if so, how does one evaluate it?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.