Posting a problem i have to solve, just trying to understand how bayes works on conditional probabiliries

(a) Assume X follows a Poisson distribution P(X = x|λ) = e^−λ * λ^x / x! , where the parameter λ follows the gamma distribution with parameters α and β. Using the Bayes’ rule, write down the conditional probability density p(λ|X = x) for some observed value of x (that is, the posterior distribution of the parameter conditional on having observed the data instance). Hint: Start from the definitions of the probability density functions and simplify the expression. (b) Derive the posterior distribution obtained after having seen a collection of N independent observations {x1,...,xN}.

So, using the bayes rule i have P(X = x|λ) = P(λ|X = x)*P(X = x) /P(λ). (1)

λ follows gamma so P(λ) = (β^α*λ^(a-1)*e^(-bλ))/(a-1)! (2)

To find P(X = x) i integrate P(X = x|λ) from 0 to infinite with dλ and i get

P(X = x) = -Γ(a+1, λ)/x! (3)

P(X = x|λ) = e^−λ * λ^x / x! is given (4)

So, to compute P(λ|X = x) i use (2), (3), (4) on (1) and i ended up to

P(λ|X = x) = e^(-λ(b+1)) * λ^(x+a-1) / (Γ(a+1, λ) (a-1)!)

Im new to these concepts so feel free to note the points that dont make sense in my results or concepts that i need better understanding of.

Also, i would appreciate if someone explains what is needed in (b), thanks!


You don't need to compute $\mathbb{P}(X)$. Poisson and Gamma random variables are conjugate, which means that if Gamma is prior, Poisson is likelihood, then posterior is also Gamma.

What you want to do in a) is:

$$p(\lambda \lvert X) \propto p(X \lvert \lambda) p(\lambda \lvert \alpha, \beta)$$

from Bayes rule. We ignore $\mathbb{P}(X)$ since it's constant and we can infer normalization factor from shape of distribution. It's possible, because shape of distribution is known (in this case, Gamma), so we can just look it up.

So (slightly abusing notation): $$p(X = k \lvert \lambda ) = \exp(-\lambda) \frac{\lambda^k}{k!}$$ $$p(\lambda \lvert \alpha, \beta) = \frac{\beta^{\alpha}}{\Gamma{\alpha}} \lambda^{\alpha - 1} \exp(-\lambda \beta)$$

Hence: $$p(\lambda \lvert X) p(X) = \exp(-\lambda (\beta + 1)) \lambda^{k + \alpha - 1} \frac{\beta^{\alpha}}{\Gamma({\alpha})} \propto \exp(-\lambda (\beta + 1)) \lambda^{k + \alpha - 1} $$

We recognize expression on the far right as the shape of new gamma distribution: $G(k+\alpha, \beta+1)$. Norming constant is $(\beta+1)^{k+\alpha} / \Gamma(k+\alpha)$

So finally we arrive at: $$\lambda \lvert X \sim Gamma(\lambda \lvert k+\alpha, \beta+1)$$

Regarding b) - let $X = \{X_1, X_2, \dots, X_n\}$ be your sample. Then $p(X \lvert \lambda) = \prod p(X_i \lvert \lambda)$, since $X_i$ are iid. Proof follows in the indentical manner from here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.