I'm a beginner student in Bayesian data analysis and I'm trying to understand how the posterior distribution of the Beta distribution is derived. In many of the references I've been reading an example of the posterior of the Beta distribution is presented in the following form:

$$p(\theta\mid\alpha, \beta,y) \propto p(y\mid\alpha,\beta,\theta)\;p(\theta\mid\alpha,\beta),$$

where $y \sim \operatorname{Bin}(n, \theta)$ and $\theta \sim Beta(\alpha, \beta)$. When this form is presented in the tutorials it is briefly mentioned that the normalizing denominator can be "forgotten" and I can't recall a presentation of this posterior function where the normalizing factor is included in the equation.

I tried to figure this out myself:

$$p(\theta\mid\alpha, \beta,y) = \frac{p(\theta, \alpha, \beta, y)}{p(\alpha, \beta, y)}$$

$$p(y\mid\alpha, \beta, \theta) = \frac{p(\theta, \alpha, \beta, y)}{p(\alpha, \beta, \theta)}$$

$$p(\theta\mid\alpha, \beta,y)\;p(\alpha, \beta, y) = p(y\mid\alpha, \beta, \theta)\;p(\alpha, \beta, \theta)$$

$$p(\theta\mid\alpha, \beta,y) = \frac{p(y\mid\alpha, \beta, \theta)\;p(\alpha, \beta, \theta)}{p(\alpha, \beta, y)}= \frac{p(y\mid\alpha, \beta, \theta)\;p(\theta\mid\alpha,\beta)\;p(\alpha,\beta)}{p(y\mid\alpha,\beta)\;p(\alpha,\beta)}$$

$$=\frac{p(y\mid\alpha, \beta, \theta)\;p(\theta\mid\alpha,\beta)}{p(y\mid\alpha,\beta)}.$$

So I get pretty close to the form given in references and this makes me think that does:

$$p(y\mid\alpha, \beta)=c\in\mathbb{R},$$

that is, is the normalizing factor a constant?

Sorry if my question is unclear. This is pretty new stuff for me and I get confused easily :)

  • $\begingroup$ $p(y\mid\alpha,\beta)$ doesn't depend on $\theta$, so we consider it constant with regard to $\theta$, and thus 'ignore it' in the expression for the posterior. $\endgroup$ – Blaza Feb 6 '17 at 14:25
  • 1
    $\begingroup$ Thank you @Blaza appreciate it :) Could you perhaps post your comment as an answer? $\endgroup$ – jjepsuomi Feb 6 '17 at 15:16

The factor $p(y\mid\alpha,\beta)$ doesn't depend on $\theta$ and thus we consider it a constant with regard to $\theta$. Therefore we can 'ignore' it in the expression for the posterior (it is contained in the '$\propto$' notation).

One way it is often written is also $p(y\mid\alpha,\beta)=\int_\Theta{}p(y\mid\alpha,\beta,\theta)p(\theta\mid\alpha,\beta)d\theta$. It's basically the numerator averaged over all values of $\theta$. That's also a reason why it is called the normalising factor, it ensures the posterior integrates to $1$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.