Given the multinomial distribution: $$P[X_1 = x_1,\ldots,X_k = x_k|n,\theta_1,\ldots,\theta_k] = \frac{n!}{x_1!\ldots x_k!} \prod_{i = 1}^k \theta_i^x$$ where $\sum_{i = 1}^k \theta_i = 1,\sum_{i = 1}^k x_i = n$ and a sample $x = (x_1,\ldots,x_k)$

I want to compute the posterior mean of $\theta_1$ using the conjugate family.

My reasoning

The multinomial distribution is $p(x|\theta) = \frac{n!}{\prod_i x_i!} \prod_i \theta_i^{x_i}$ and the Dirichlet distribution is $p(\theta|\alpha) = \frac{1}{B(\alpha)}\prod_i \theta_i^{\alpha_i-1}$. Now I should compute the posterior distribution which seems to be

$$p(x|\theta) p(\theta|\alpha) \propto \prod_i \theta_i^{x_i+\alpha_i-1}$$

So I can assume that the posterior $p(\theta|x,\alpha')$ is a Dirichlet distribution with parameters $x_i+\alpha_i$.

The posterior mean would be $\int_{\theta} \theta p(\theta|x,\alpha') d\theta$

My question

How does one finish this exercise, namely how does one integrate the last expression?

  • 1
    $\begingroup$ I haven't worked with trying to find the Bayesian posterior for this, but I asked a similar question which may help with the integration. $\endgroup$ – Clarinetist Jan 9 '18 at 19:41
  • $\begingroup$ If you know the formula for the norming constant of a Dirichlet distrib, you should be able to read the answer off of the ratio of two norming constants. $\endgroup$ – kimchi lover Jan 9 '18 at 22:34

The posterior distribution is a distribution in $\theta$ with parameters $x_i + \alpha_i$. This amounts to compute the mean of a particular Dirichlet distribution. This is done here. Therefore: $E_{\pi(\theta|x)} \theta_j = \frac{x_j + \alpha_j}{\sum (x_i+\alpha_i)} = \frac{x_j + \alpha_j}{n + \sum \alpha_i}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.