2
$\begingroup$

Suppose that a random sample $X_1, ..., X_n$ is taken from a distribution where $X$ is a random variable that follows a Poisson distribution with mean $w$. Furthermore, $w$ is modeled with a random variable $W$ that follows a chi-square distribution with $r = 4$ degrees of freedom. (i) Determine the posterior distribution of $W$ given that $X_1 = x_1, X_2 = x_2, ..., X_n = x_n$ (ii) Find the Bayesian estimator for $W$ that minimizes the mean-square error (iii) Before the sample was taken, Jack tells you that $w$ lies in $(0, 1)$, however, no further information is provided about which values $w$ would most likely take in this interval. Do you pick a new prior pdf for $W$? Justify your answer, stating what new prior pdf you would take if so, or if not, why you would not pick a new prior pdf.

Hint: Integration is unnecessary. The posterior distribution should be familiar if you use proportionality! You should use this posterior distribution to find the mean-square error.

My attempt:

(i) From chi-square with $r = 4$, the prior pdf of $W$ is

$$f_W(w) = \frac{we^{-w/2}}{4}$$

The pdf of a Poisson with mean $w$ is:

$$f(x|w) = \frac{w^{x}e^{-w}}{x!}$$

The joint pdf of $X_1, X_2, ..., X_n$ is:

$$F(x_1, ..., x_n|w) = \frac{w^{\sum{x_i}}e^{-nw}}{\prod{x_i!}}$$

The posterior probability density function (pdf) of $W$ given $X_1 = x_1, X_2 = x_2, ..., X_n = x_n$ is:

$$f(w|x) = \frac{F(x_1, ..., x_n|w) \cdot f_W(w)}{f_X(x)} = \frac{F(x_1, ..., x_n|w) \cdot f_W(w)}{\int_{-\infty}^{\infty} F(x_1, ..., x_n|w) \cdot f_W(w) \,dw}$$

We apply proportionality:

$$\frac{F(x_1, ..., x_n|w) \cdot f_W(w)}{\int_{-\infty}^{\infty} F(x_1, ..., x_n|w) \cdot f_W(w) \,dw} \propto F(x_1, ..., x_n|w) \cdot f_W(w) = \frac{w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}}{4\prod{x_i!}}$$

From here, there are a few possible posteriors to choose from: Chi-square, Exponential or Gamma.

By mere inspection:

If Chi-square, we have parameter $r = 4 + 2\sum{x_i}$.

If Exponential, we have parameter $\theta = \frac{-2}{w(2n+1)}$ or $\theta = \frac{1}{w^{1+\sum{x_i}}}$.

If Gamma, we have parameters $\alpha = 2 + \sum{x_i}$ and $\theta = \frac{2}{2n+1}$.

I am unsure which one is the correct posterior distribution as they all seem likely.

(ii) Depends on (i) but am I supposed to use the fact that $E(W|X)$ is the least mean square here?

(iii) Not sure.

Any assistance especially with (i) and (iii) is much appreciated.

$\endgroup$
7
  • $\begingroup$ For (i), it is obviously not exponential since there cannot be a $w$ term in it apart from the exponential. Note also that your parameter $\theta$ must be a constant for an exponential distribution and cannot depend on the variable of interest ($w$), and furthermore, it is required that $\theta > 0$. It also can't be chi-squared because the exponent on $e$ must be equal to $-w/2$. Thus, Gamma is probably correct, but note: for Gamma, your $\theta$ needs to be $> 0$. Check your algebra and compare with the Gamma PDF. $\endgroup$ May 19, 2021 at 20:47
  • $\begingroup$ Are you defining an exponential distribution with parameter $\theta$ to have density function under definition 1: $f(x) = \theta e^{-\theta x}$, $x > 0$ - or definition 2: $f(x) = \dfrac{1}{\theta}e^{-x/\theta}$, $x > 0$? $\endgroup$ May 19, 2021 at 20:56
  • $\begingroup$ @Clarinetist Thank you for the response. The second definition. $\endgroup$ May 19, 2021 at 21:01
  • 2
    $\begingroup$ For (iii) you might consider a uniform$(0, 1)$ prior. The chi squared distribution is nonzero for values far from $(0, 1)$, so might not be as good of a prior as a uniform$(0, 1)$. $\endgroup$
    – Mason
    May 19, 2021 at 21:23
  • 1
    $\begingroup$ @SupremePickle chi-squared $r$ distribution is the distribution of the sum of squares of $r$ i.i.d. normal$(0, 1)$ variables. So the chi-squared $r$ density is nonzero for values outside of $(0, 1)$. See the graphs here: en.wikipedia.org/wiki/Chi-square_distribution $\endgroup$
    – Mason
    May 19, 2021 at 21:37

1 Answer 1

3
$\begingroup$

For (i), you must be careful of the assumptions made in the definitions of the density functions.

For the exponential distribution, we have the density function $$f_{X}(x) = \dfrac{1}{\theta} e^{- x/\theta}\text{, }\qquad x > 0$$ with $\theta > 0$ a constant - i.e., it cannot depend on $x$.

For the Gamma distribution, we have the density function $$f_{X}(x) = \dfrac{1}{\Gamma(\alpha)\theta^\alpha}x^{\alpha - 1}e^{-x/\theta}\text{, }\qquad x > 0$$ with $\alpha > 0$ and $\theta > 0$ constants - again, not depending on $x$.

The Chi-squared distribution with $\nu$ degrees of freedom is just the Gamma distribution with $\alpha = \nu/2$ and $\theta = 2$.

As far as I can tell, your computation of the posterior PDF is correct: we know that $$f(w \mid x) \propto \frac{w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}}{4\prod{x_i!}}$$ but we can take this one step further: note that $w$ is the variable of interest, so we can clean this up to end up with $$f(w \mid x) \propto \frac{w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}}{4\prod{x_i!}} \propto w^{1+\sum{x_i}}e^{-nw-\frac{w}{2}}\text{.}$$ Now, we observe that this is indeed similar to the exponential, Gamma, and Chi-squared distributions. However, for all three of these, we must separate the variable of interest in the exponent in the numerator - i.e., this suggests $$f(w \mid x) \propto w^{1+\sum{x_i}}e^{-w\left(n+\frac{1}{2}\right)}\text{.}$$ This has a $w$ term in it, which would not be the case for the exponential distribution (to match the notation of the exponential PDF above, replace $w$ with $x$). It is also not Chi-squared, since the factor of the exponent of $e$ should be $1/2$. So it's probably Gamma, and we can see this by writing $$-w\left(n + \dfrac{1}{2}\right) = -\dfrac{w}{\left(n + \dfrac{1}{2}\right)^{-1}}$$ which yields $\theta = \left(n + \dfrac{1}{2}\right)^{-1} > 0$ and $\alpha = 1 + \sum x_i + 1 = 2 + \sum x_i > 0$, thus we have a Gamma PDF.

Your thought for (ii) is correct; given (i), this should be easy to find since you know the posterior distribution.

I don't have enough experience with Bayesian model fitting to answer (iii).

$\endgroup$
4
  • $\begingroup$ This makes a lot of sense, thanks! Minor error though: you forgot to factor out the negative earlier on. $\endgroup$ May 19, 2021 at 21:19
  • $\begingroup$ @SupremePickle Ah, thanks! Editing now. $\endgroup$ May 19, 2021 at 21:20
  • $\begingroup$ Sorry but I had a follow-up question: Can't we factor out the $\frac{1}{2}$ in the exponent as follows: $-\frac{1}{2}w(2n+1)$? $\endgroup$ May 20, 2021 at 0:11
  • $\begingroup$ @SupremePickle Sure, but $w(2n+1)$ is not the variable of interest; you can't say it is Chi-squared on that basis. $\endgroup$ May 20, 2021 at 1:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .