I'm trying to follow a Bayesian inference lecture and got stuck on a simplification. In the question, I'm trying to find the posterior distribution, given

$$X_1 ,\ldots, X_n \mid \lambda \stackrel{\text{iid}}{\sim} \operatorname{Poisson}(\lambda),$$

$$\lambda \sim \operatorname{gamma}(\alpha,\beta)$$

I understand the use of conjugates Etc. to make the integral tractable. However, I'm getting stuck on the last part of the simplification. Could someone walk me through the simplification of this expression:

$$\frac {\Gamma(n\bar{x} + \alpha)}{\Gamma{(\alpha})\prod_{i=1}^n x_i!} \left[\frac{\beta}{n + \beta}\right]^\alpha\left[\frac{1}{n + \beta} \right]^{x_i} \cdots \left[\frac{1}{n + \beta}\right]^{x_n} $$

to the given posterior distribution $\lambda \mid x_1, \ldots, x_n \sim \operatorname{gamma}(n\bar{x} + \alpha, n + \beta)$ ?

Thank you!


The posterior is the following measure multiplied by a normalizing constant: $$ \overbrace{(\ell^{x_1} e^{-\ell}) \cdots (\ell^{x_n} e^{-\ell})}^\text{likelihood} {} \cdot {} \overbrace{ (\beta\ell)^{\alpha-1} e^{-\beta\ell} (\beta\,d\ell) }^\text{gamma prior} $$ I've omitted the $\text{“}x_i!\text{''s}$ in the denominators since they don't depend on $\ell$ and similarly the $\Gamma(\alpha)$ in the denominator. The above is proportional, as a function of $\ell,$ to $$ \propto \quad \ell^{x_1+\cdots +x_n +\alpha-1} e^{-(n+\beta)\ell} \, d\ell. \tag 1 $$ This is gamma distribution, with $x_1+\cdots+x_n+\alpha$ in the posterior where $\alpha$ appeared in the prior, and with $n+\beta$ in the posterior where $\beta$ appeared in the prior.

The expression you're trying to simplify looks as if it would appear in an attempt to find the normalizing constant. I wouldn't worry about that until after you've reached line $(1)$ above. In line $(1),$ you've got a gamma distribution but the normalizing constant is not specified. But it is a known function of the two parameters $x_1+\cdots+x_n+\alpha$ and $n+\beta.$

  • $\begingroup$ Thank you for your answer. Yes, I believe you are correct that I am solving for the normalization term. We went about it by solving the marginal distribution integral in the denominator. This was the last step shown before the final solution. I get the sense from your answer that there is more groundwork I need to understand as I am not familiar with your solution. Is this just a method of understanding solutions via conjugates? Do you have a recommended text or source for this material? I'm interested to learn more. $\endgroup$ Nov 19 '19 at 16:31
  • $\begingroup$ @washedupengineer : Here's a start: Suppose the prior distribution is $$\Big( \Pr(A), \Pr(B), \Pr(C)\Big) = \left. \big( 4, 5, 11\big)\right/20$$ and the likelihood is $$\Big( \Pr(D\mid A),\, \Pr(D\mid B),\, \Pr(D\mid C) \Big) = \big( 9/10,\, 8/10,\, 7/10\big).$$ Then you have the posterior distribution $$ \begin{align} & \Big(\Pr(A\mid D),\, \Pr(B\mid D),\, \Pr(C\mid D) \Big) \\ {} \\ \propto {} & (4,5,11)\times(9,8,7) \\ {} \\ \propto {} & (36,\, 40,\, 77) \\ {} \\ \propto {} & (36,\, 40,\, 77)/153 \\ {} \\ = {} & \Big( \Pr(A\mid D),\, \Pr(B\mid D),\, \Pr(C\mid D) \Big) \end{align} $$ $\endgroup$ Nov 20 '19 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.