So, I understand the idea of likelihood when it comes to throwing dice. For example:

Denote two heads in two tosses HH. Assuming that each successive coin flip is i.i.d., then the probability of observing HH is

$$ P({HH}\mid p_{\text{H}}=0.5)=0.5^2=0.25.$$

But when it comes to models I get really confused. To calculate the Bayes factor, we can use Bayes rule to find the likelihood of model $M,$ given the data $D,$ in this way: $$P(M\mid D)= \frac{P(D\mid M)×P(M)}{P(D)}$$

But what does this mean intuitively? Likelihood of a model doesn't make sense to me.

Another question: If I have two models $M_1$ and $M_2$ and $M_1 \subseteq M_2 $ meaning that $M_2$ extend $M_1$, then we must always have that $P(M_2\mid D) \geq P(M_1\mid D)$, right? But then the probability odds $$\frac{P(M_1\mid D)}{P(M_2\mid D)} < 1$$

What do I miss? I hope you can clarify me a bit. All answers are appreciated. Thanks in advance.


In this context the words "probability" and "likelihood" often mean two different things.

Suppose the coin turns up "heads" $100\times p\%$ of the time.

Suppose $M_1$ is the statement that $0.49<p<0.51$ and $M_2$ says $0.2<p<0.8.$ Then one must have $\Pr(M_1)\le \Pr(M_2).$

The probability, given the value of $p,$ of the outcome $HH$ is $$ \Pr(HH\mid p) = p^2. $$ If that is regarded as a function of $p,$ then it is what is called a likelihood function: $L(p) = p^2.$

Suppose $f(r)\, dr,$ for $0<r<1,$ is the probability distribution of $p.$ That implies, for example, that $\displaystyle \Pr(0.3<p<0.55) = \int_{0.3}^{0.55} f(r)\,dr,$ and similarly for other intervals. Then $$ \Pr(0.3<p<0.55\mid HH) = c\int_{0.3}^{0.55} L(r) f(r)\,dr = c\int_{0.3}^{0.55} r^2 f(r)\, dr $$ where $c$ is the normalizing constant, which satisfies $$ c\int_0^1 r^2 f(r)\, dr = 1. $$

  • $\begingroup$ Allright. So I understand the $Pr(M)$. But what about $Pr(M|D)$? You clarify the condtional expectation of the outcome HH very good. But could you clarify what we mean by the conditional expectation of a statement/model? $\endgroup$ – LocalMartingale Oct 25 '20 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.