1
$\begingroup$

For the following problem, I’m stuck at the 3rd question. I would appreciate if you could validate my answer to the 1st question as well.

Problem

Let $X_1, \dots, X_n \sim \operatorname{Gamma}(2, \theta)$ with support $(0, \infty)$ and p.d.f. $$f(x | \theta) = \theta^2 x e^{-\theta x}$$ Let $\operatorname{Gamma}(2, \lambda)$ where $\lambda > 0$ be the prior distribution of $\theta$ with support $(0, \infty)$ and p.d.f. $$g(\theta) = \lambda^2 \theta e^{-\lambda\theta}$$

  1. Identify the posterior density function $g(\theta | x_1, \dots, x_n)$ of $\theta$ given data $X_1 = x_1, \dots, X_n = x_n$.
  2. Determine the Bayesian estimator $\hat\theta$ of $\theta$ under the square loss function.
  3. Determine the predictive density of $X_{n+1}$ given $X_1 = x_1, \dots, X_n = x_n$, namely $f(x_{n+1} | x_1, \dots, x_n)$.

Answer

  1. According to the Bayes theorem, \begin{align*} g(\theta | x_1, \dots, x_n) &= \frac{f(x_1, \dots, x_n | \theta) g(\theta)}{f(x_1, \dots, x_n)} = g(\theta) \frac{\prod_{i=1}^n f(x_i | \theta)}{\prod_{i=1}^n f(x_i)} \end{align*} The marginal density of the data is hard to determine. It would likely require three integrations by parts. $$ f(x_i) = \int_0^\infty {f(x_i | \theta) g(\theta)}{d\theta} = \int_0^\infty {\lambda^2 x_i \theta^3 e^{-(\lambda+x_i)\theta}}{d\theta} $$ However, it is constant w.r.t. to $\theta$, so we can write \begin{align*} g(\theta | x_1, \dots, x_n) &\propto f(x_1, \dots, x_n | \theta) g(\theta) \\&= \left(\prod_{i=1}^n \theta^2 x_i e^{-\theta x_i} \right) \lambda^2 \theta e^{-\lambda \theta} \\&\propto \theta^{2n+1} e^{-(\lambda + n \bar{x}) \theta} \end{align*} It seems to be the p.d.f of $\operatorname{Gamma}(\alpha = 2(n+1), \beta = \lambda+n\bar{x})$, that is \begin{align*} g(\theta | x_1, \dots, x_n) &= \frac{\beta^\alpha}{\Gamma(\alpha)} \theta^{\alpha-1} e^{-\beta \theta} = \frac{(\lambda+n\bar{x})^{2(n+1)}}{(2n+1)!} \theta^{2n+1} e^{-(\lambda+n\bar{x}) \theta} \end{align*} Note: That exchange seems to validate the approach.

  2. The Bayes estimator minimizes the posterior expected loss. For the squared loss, it is \begin{align*} \hat\theta(x_1, \dots, x_n) &= \mathbb{E}[{\theta | X_1 = x_1, \dots, X_n = x_n}] \end{align*} The expectation of $\operatorname{Gamma}(\alpha, \beta)$ is $\alpha / \beta$, so $$\hat\theta(x_1, \dots, x_n) = \frac{2(n+1)}{\lambda + n \bar{x}}$$

  3. The predictive density is \begin{align*} f(x_{n+1} | x_1, \dots, x_n) &= \int_0^\infty {f(x_{n+1} | \theta) g(\theta | x_1, \dots, x_n)} d\theta \\&= \int_0^\infty{\theta^2 x_{n+1} e^{-\theta x_{n+1}} \cdot \frac{(\lambda + n\bar{x})^{2(n+1)}}{(2n+1)!} \theta^{2n+1} e^{-(\lambda + n \bar{x}) \theta}} d\theta \\&= \frac{x_{n+1} (\lambda + n\bar{x})^{2(n+1)}}{(2n+1)!} \int_0^\infty {\theta^{2(n+1)+1} e^{-(\lambda + n \bar{x} + x_{n+1}) \theta}} d\theta \\&= \mathbf{?} \end{align*} The function inside the integral seems to be $k$ times the p.d.f. of $\operatorname{Gamma}(2(n+2), \lambda + n \bar{x} + x_{n+1})$ with $k > 0$. Should I go on with this idea, noticing that the integral should equal a constant (total probabilities w.r.t. $\theta$), and find the $k$ that makes everything coherent? Or are there some conjugacy properties to leverage?

Thank you very much for your help.

$\endgroup$
1
  • 1
    $\begingroup$ I think you can "go on with this idea". Alternatively, Wikipedia seems to suggest that with a Gamma likelihood where the shape is known (here $2$) and a Gamma prior, the predictive distribution is a generalised Beta prime / compound Gamma distribution but I doubt that would be better. $\endgroup$
    – Henry
    Commented Jan 22 at 12:46

1 Answer 1

1
$\begingroup$

I think I figured it out.

Let $\alpha_k = 2(k+1)$ and $\beta_k = \lambda + k \bar x_k = \lambda + \sum_{i=1}^n x_k$. The predictive density is \begin{align*} f(x_{n+1} | x_1, \dots, x_n) &= \int_0^\infty {f(x_{n+1} | \theta) g(\theta | x_1, \dots, x_n)}{d\theta} \\&= \int_0^\infty {\theta^2 x_{n+1} e^{-\theta x_{n+1}} \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} \theta^{\alpha_n-1} e^{-\beta_n \theta}}{d\theta} \\&= \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} x_{n+1} \int_0^\infty {\theta^{2(n+1)+1} e^{-(\lambda + n \bar{x}_n + x_{n+1}) \theta}}{d\theta} \\&= x_{n+1} \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} \int_0^\infty {\theta^{\alpha_{n+1}-1} e^{-\beta_{n+1} \theta}}{d\theta} \end{align*} The expression inside the integral has the shape of the p.d.f. of $\operatorname{Gamma}(\alpha_{n+1}, \beta_{n+1})$: \begin{align*} h(\theta | x_1, \dots, x_{n+1}) = \frac{\beta_{n+1}^{\alpha_{n+1}}}{(\alpha_{n+1}-1)!} \theta^{\alpha_{n+1}-1} e^{-\beta_{n+1} \theta} \end{align*} By the law of total probabilities, \begin{align*} \int_0^\infty {h(\theta | x_1, \dots, x_{n+1})}{d\theta} &= 1 \\ \iff \int_0^\infty{\theta^{\alpha_{n+1}-1} e^{-\beta_{n+1} \theta}}{d\theta} &= \frac{(\alpha_{n+1}-1)!}{\beta_{n+1}^{\alpha_{n+1}}} \\\\ \implies f(x_{n+1} | x_1, \dots, x_n) &= x_{n+1} \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} \frac{(\alpha_{n+1}-1)!}{\beta_{n+1}^{\alpha_{n+1}}} \\&= \frac{(\alpha_{n+1}-1)!}{(\alpha_n-1)!} \frac{\beta_n^{\alpha_n}}{\beta_{n+1}^{\alpha_{n+1}}} x_{n+1} \\&= \frac{(2n+3)!}{(2n+1)!} \frac{(\lambda + n\bar{x}_n)^{2(n+1)}}{(\lambda + (n+1) \bar x_{n+1})^{2(n+2)}} x_{n+1} \\&= (2n+2)(2n+3) \left( \frac{\lambda + n\bar{x}_n}{\lambda + (n+1) \bar x_{n+1}} \right)^{2(n+1)} \frac{x_{n+1}}{(\lambda + (n+1) \bar x_{n+1})^2} \\&= \frac{(2n+2)(2n+3) x_{n+1}}{(\lambda + (n+1) \bar x_{n+1})^2} \left( 1 - \frac{x_{n+1}}{\lambda + (n+1) \bar x_{n+1}} \right)^{2(n+1)} \end{align*}

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .