Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Let's say I have some data, for example $d = 0.112$. And I have a known model $m$ which just produced uniformly distributed values over the interval $[0,.5]$. I am interested in computing the likelihood of my model given the data, in the Bayesian sense, i.e. $P(D=d | M = m)$.

What is this? is it $0$?

Given $m$ is a continuous distribution, I can't see how it could be anything other than likelihood 0? More generally, I can't see how the likelihood with any continuous model could be anything other than 0. I could imagine using the probability density function, but to be honest I am not entirely sure what this would mean, and it would give a likelihood of 2, which is greater than 1 and hence not a probability.

Thanks

share|improve this question
    
Informally, we are in a "$0/0$" situation. More precisely, we look at intervals of small positive length, and then let that length approach $0$. –  André Nicolas Feb 4 '12 at 4:14
add comment

1 Answer

up vote 3 down vote accepted

The simplest answer is that a likelihood function $\mathcal{L}$ is not a probability, which is why different names are used. In particular it does not need to sum or integrate to $1$.

For your uniform distribution, all of the values in $[0,0.5]$ are equally likely, so you want your calculation to produce the same likelihood for each of them. If you use the density function then you will get $2$ for all of them, but it would not matter if you came up with $10$ or any other constant positive number: it is the relative likelihood that matters.

This becomes even more obvious in Bayesian methods. If the prior distribution for the parameter $M$ is $\pi_0(m)$ and you use $$\pi(m|D=d) = \frac{\pi_0(m) \mathcal{L}_{D=d}(M=m) }{ \int_n \pi_0(n) \mathcal{L}_{D=d}(M=n) \,dn}$$ to calculate the posterior distribution for $M$ then it should be obvious that multiplying the likelihood function by a non-zero constant cancels out in the calculation of the posterior distribution.

share|improve this answer
    
Thanks. But then let's say our model is more complex and can be represented as a probabilistic program. For example: IF flip-coin() = true THEN sample from uniform(0,.5) ELSE sample from normal_distribution(.1,.2). When trying to compute the likelihood of some datapoint, it seems plausable that we'd compute the probability of the if-statement returning true (0.5) multiplied by probability of uniform dist, + probability of not being true * probability of being sampled from normal. Is this a valid thing to do when using likelihood functions which are not probabilites? –  zenna Feb 6 '12 at 17:11
    
@zenna: you haven't actually got a parameter of interest in your example. So let's suppose that your example has probability $m$ of being uniformly distributed on $[0,0.5]$ and $1-m$ of being normally distributed with mean $0.1$ and variance $0.2$; you then observe the value $0.3$. $\mathcal{L}(m)$, the likelihood of different values of $m$ given this data, is (proportional to) $2m+\dfrac{1-m}{\sqrt{0.4\, \pi }}\, e^{-0.1}$ –  Henry Feb 6 '12 at 17:43
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.