1
$\begingroup$

I am trying to work with the Beta Binomial model for Machine Learning and Bayesian probability theory.

Here we have the Likelihood for the Beta-Binomial model:

Beta_Binomial_likelihood

Here we have the Prior:

Beta_Binomial_prior

And if we were to multiply them, we would get this:

posterior_1

My first question is this: The textbook says that the prior distribution is proportional to the likelihood times the prior itself:

proportional_prior

But how is this possible? The $p(D | \theta)$ term is NOT CONSTANT - it varies as $\theta$ varies (and NOT necessarily proportionally with $p(\theta)$), correct?

And my second question is this: The textbook also says that the posterior is this:

proportional_posterior

I've already tried to derive this, but no luck. Could someone give me a couple steps in the right direction for this as well? Where do the 1 Bin and 2 Beta terms come from?

Thanks in advance

$\endgroup$
2
$\begingroup$

I would have expected the book to say the posterior is proportional to the likelihood times the prior so in your notation $$p(\theta \mid \mathcal{D}) \propto p(\mathcal{D}\mid \theta)\,p( \theta)$$ and your final line seems to be missing a symbol

$$p(\theta \mid \mathcal{D}) \propto \text{Bin}(N_1\mid \theta, N_0+N_1) \, \text{Beta}(\theta \mid a,b) \propto \text{Beta}(\theta \mid N_1+a,N_0+b) $$

with

  • the binomial $p(\mathcal{D}\mid \theta)=\text{Bin}(N_1\mid \theta, N_0+N_1)$ being proportional to the likelihood and
  • the Beta $p( \theta) =\text{Beta}(\theta \mid a,b)$ being the conjugate prior for $\theta$ and
  • the Beta $p(\theta \mid \mathcal{D}) = \text{Beta}(\theta \mid N_1+a,N_0+b) $ being the posterior distribution for $\theta$
$\endgroup$
  • $\begingroup$ Hey Henry, the book says exactly what I put in my first question (that was an unedited screenshot). Was it a typo? It does not make any sense $\endgroup$ – Josh Jul 18 '17 at 19:15
  • 1
    $\begingroup$ @Josh Does what I have written make sense? Could the book have typos like this? $\endgroup$ – Henry Jul 18 '17 at 19:17
  • $\begingroup$ What you have written makes much more sense than the textbook. I'm not sure whether or not the book is likely to have these kinds of typos - and the fact that I'm not sure is what's bothering me. And also, every one of the images that you see in my question were screenshots, so when you added that symbol you said was missing, it looks as if you uncovered yet another typo $\endgroup$ – Josh Jul 18 '17 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.