Bayesian statistics to find a distribution Suppose that a random sample $X_1, ..., X_n$ is taken from a distribution where $X$ is a random variable that follows a Poisson distribution with mean $w$. Furthermore, $w$ is modeled with a random variable $W$ that follows a chi-square distribution with $r = 4$ degrees of freedom. (i) Determine the posterior distribution of $W$ given that $X_1 = x_1, X_2 = x_2, ..., X_n = x_n$ (ii) Find the Bayesian estimator for $W$ that minimizes the mean-square error (iii) Before the sample was taken, Jack tells you that $w$ lies in $(0, 1)$, however, no further information is provided about which values $w$ would most likely take in this interval. Do you pick a new prior pdf for $W$? Justify your answer, stating what new prior pdf you would take if so, or if not, why you would not pick a new prior pdf.
Hint: Integration is unnecessary. The posterior distribution should be familiar if you use proportionality! You should use this posterior distribution to find the mean-square error.
My attempt:
(i) From chi-square with $r = 4$, the prior pdf of $W$ is
$$f_W(w) = \frac{we^{-w/2}}{4}$$
The pdf of a Poisson with mean $w$ is:
$$f(x|w) = \frac{w^{x}e^{-w}}{x!}$$
The joint pdf of $X_1, X_2, ..., X_n$ is:
$$F(x_1, ..., x_n|w) = \frac{w^{\sum{x_i}}e^{-nw}}{\prod{x_i!}}$$
The posterior probability density function (pdf) of $W$ given $X_1 = x_1, X_2 = x_2, ..., X_n = x_n$ is:
$$f(w|x) = \frac{F(x_1, ..., x_n|w) \cdot f_W(w)}{f_X(x)} = \frac{F(x_1, ..., x_n|w) \cdot f_W(w)}{\int_{-\infty}^{\infty} F(x_1, ..., x_n|w) \cdot f_W(w) \,dw}$$
We apply proportionality:
$$\frac{F(x_1, ..., x_n|w) \cdot f_W(w)}{\int_{-\infty}^{\infty} F(x_1, ..., x_n|w) \cdot f_W(w) \,dw} \propto F(x_1, ..., x_n|w) \cdot f_W(w) = \frac{w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}}{4\prod{x_i!}}$$
From here, there are a few possible posteriors to choose from: Chi-square, Exponential or Gamma.
By mere inspection:
If Chi-square, we have parameter $r = 4 + 2\sum{x_i}$.
If Exponential, we have parameter $\theta = \frac{-2}{w(2n+1)}$ or $\theta = \frac{1}{w^{1+\sum{x_i}}}$.
If Gamma, we have parameters $\alpha = 2 + \sum{x_i}$ and $\theta = \frac{2}{2n+1}$.
I am unsure which one is the correct posterior distribution as they all seem likely.
(ii) Depends on (i) but am I supposed to use the fact that $E(W|X)$ is the least mean square here?
(iii) Not sure.
Any assistance especially with (i) and (iii) is much appreciated.
 A: For (i), you must be careful of the assumptions made in the definitions of the density functions.
For the exponential distribution, we have the density function
$$f_{X}(x) = \dfrac{1}{\theta} e^{- x/\theta}\text{, }\qquad x > 0$$
with $\theta > 0$ a constant - i.e., it cannot depend on $x$.
For the Gamma distribution, we have the density function
$$f_{X}(x) = \dfrac{1}{\Gamma(\alpha)\theta^\alpha}x^{\alpha - 1}e^{-x/\theta}\text{, }\qquad x > 0$$
with $\alpha > 0$ and $\theta > 0$ constants - again, not depending on $x$.
The Chi-squared distribution with $\nu$ degrees of freedom is just the Gamma distribution with $\alpha = \nu/2$ and $\theta = 2$.
As far as I can tell, your computation of the posterior PDF is correct: we know that
$$f(w \mid x) \propto \frac{w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}}{4\prod{x_i!}}$$
but we can take this one step further: note that $w$ is the variable of interest, so we can clean this up to end up with
$$f(w \mid x) \propto \frac{w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}}{4\prod{x_i!}} \propto w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}\text{.}$$
Now, we observe that this is indeed similar to the exponential, Gamma, and Chi-squared distributions. However, for all three of these, we must separate the variable of interest in the exponent in the numerator - i.e., this suggests
$$f(w \mid x) \propto w^{1+\sum{x_i}}e^{-w\left(n+\frac{1}{2}\right)}\text{.}$$
This has a $w$ term in it, which would not be the case for the exponential distribution (to match the notation of the exponential PDF above, replace $w$ with $x$). It is also not Chi-squared, since the factor of the exponent of $e$ should be $1/2$. So it's probably Gamma, and we can see this by writing
$$-w\left(n + \dfrac{1}{2}\right) = -\dfrac{w}{\left(n + \dfrac{1}{2}\right)^{-1}}$$
which yields $\theta = \left(n + \dfrac{1}{2}\right)^{-1} > 0$ and $\alpha = 1 + \sum x_i + 1 = 2 + \sum x_i > 0$, thus we have a Gamma PDF.
Your thought for (ii) is correct; given (i), this should be easy to find since you know the posterior distribution.
I don't have enough experience with Bayesian model fitting to answer (iii).
