# Predictive distribution for gamma likelihood and prior

For the following problem, I’m stuck at the 3rd question. I would appreciate if you could validate my answer to the 1st question as well.

#### Problem

Let $$X_1, \dots, X_n \sim \operatorname{Gamma}(2, \theta)$$ with support $$(0, \infty)$$ and p.d.f. $$f(x | \theta) = \theta^2 x e^{-\theta x}$$ Let $$\operatorname{Gamma}(2, \lambda)$$ where $$\lambda > 0$$ be the prior distribution of $$\theta$$ with support $$(0, \infty)$$ and p.d.f. $$g(\theta) = \lambda^2 \theta e^{-\lambda\theta}$$

1. Identify the posterior density function $$g(\theta | x_1, \dots, x_n)$$ of $$\theta$$ given data $$X_1 = x_1, \dots, X_n = x_n$$.
2. Determine the Bayesian estimator $$\hat\theta$$ of $$\theta$$ under the square loss function.
3. Determine the predictive density of $$X_{n+1}$$ given $$X_1 = x_1, \dots, X_n = x_n$$, namely $$f(x_{n+1} | x_1, \dots, x_n)$$.

1. According to the Bayes theorem, \begin{align*} g(\theta | x_1, \dots, x_n) &= \frac{f(x_1, \dots, x_n | \theta) g(\theta)}{f(x_1, \dots, x_n)} = g(\theta) \frac{\prod_{i=1}^n f(x_i | \theta)}{\prod_{i=1}^n f(x_i)} \end{align*} The marginal density of the data is hard to determine. It would likely require three integrations by parts. $$f(x_i) = \int_0^\infty {f(x_i | \theta) g(\theta)}{d\theta} = \int_0^\infty {\lambda^2 x_i \theta^3 e^{-(\lambda+x_i)\theta}}{d\theta}$$ However, it is constant w.r.t. to $$\theta$$, so we can write \begin{align*} g(\theta | x_1, \dots, x_n) &\propto f(x_1, \dots, x_n | \theta) g(\theta) \\&= \left(\prod_{i=1}^n \theta^2 x_i e^{-\theta x_i} \right) \lambda^2 \theta e^{-\lambda \theta} \\&\propto \theta^{2n+1} e^{-(\lambda + n \bar{x}) \theta} \end{align*} It seems to be the p.d.f of $$\operatorname{Gamma}(\alpha = 2(n+1), \beta = \lambda+n\bar{x})$$, that is \begin{align*} g(\theta | x_1, \dots, x_n) &= \frac{\beta^\alpha}{\Gamma(\alpha)} \theta^{\alpha-1} e^{-\beta \theta} = \frac{(\lambda+n\bar{x})^{2(n+1)}}{(2n+1)!} \theta^{2n+1} e^{-(\lambda+n\bar{x}) \theta} \end{align*} Note: That exchange seems to validate the approach.

2. The Bayes estimator minimizes the posterior expected loss. For the squared loss, it is \begin{align*} \hat\theta(x_1, \dots, x_n) &= \mathbb{E}[{\theta | X_1 = x_1, \dots, X_n = x_n}] \end{align*} The expectation of $$\operatorname{Gamma}(\alpha, \beta)$$ is $$\alpha / \beta$$, so $$\hat\theta(x_1, \dots, x_n) = \frac{2(n+1)}{\lambda + n \bar{x}}$$

3. The predictive density is \begin{align*} f(x_{n+1} | x_1, \dots, x_n) &= \int_0^\infty {f(x_{n+1} | \theta) g(\theta | x_1, \dots, x_n)} d\theta \\&= \int_0^\infty{\theta^2 x_{n+1} e^{-\theta x_{n+1}} \cdot \frac{(\lambda + n\bar{x})^{2(n+1)}}{(2n+1)!} \theta^{2n+1} e^{-(\lambda + n \bar{x}) \theta}} d\theta \\&= \frac{x_{n+1} (\lambda + n\bar{x})^{2(n+1)}}{(2n+1)!} \int_0^\infty {\theta^{2(n+1)+1} e^{-(\lambda + n \bar{x} + x_{n+1}) \theta}} d\theta \\&= \mathbf{?} \end{align*} The function inside the integral seems to be $$k$$ times the p.d.f. of $$\operatorname{Gamma}(2(n+2), \lambda + n \bar{x} + x_{n+1})$$ with $$k > 0$$. Should I go on with this idea, noticing that the integral should equal a constant (total probabilities w.r.t. $$\theta$$), and find the $$k$$ that makes everything coherent? Or are there some conjugacy properties to leverage?

Thank you very much for your help.

• I think you can "go on with this idea". Alternatively, Wikipedia seems to suggest that with a Gamma likelihood where the shape is known (here $2$) and a Gamma prior, the predictive distribution is a generalised Beta prime / compound Gamma distribution but I doubt that would be better. Commented Jan 22 at 12:46

Let $$\alpha_k = 2(k+1)$$ and $$\beta_k = \lambda + k \bar x_k = \lambda + \sum_{i=1}^n x_k$$. The predictive density is \begin{align*} f(x_{n+1} | x_1, \dots, x_n) &= \int_0^\infty {f(x_{n+1} | \theta) g(\theta | x_1, \dots, x_n)}{d\theta} \\&= \int_0^\infty {\theta^2 x_{n+1} e^{-\theta x_{n+1}} \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} \theta^{\alpha_n-1} e^{-\beta_n \theta}}{d\theta} \\&= \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} x_{n+1} \int_0^\infty {\theta^{2(n+1)+1} e^{-(\lambda + n \bar{x}_n + x_{n+1}) \theta}}{d\theta} \\&= x_{n+1} \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} \int_0^\infty {\theta^{\alpha_{n+1}-1} e^{-\beta_{n+1} \theta}}{d\theta} \end{align*} The expression inside the integral has the shape of the p.d.f. of $$\operatorname{Gamma}(\alpha_{n+1}, \beta_{n+1})$$: \begin{align*} h(\theta | x_1, \dots, x_{n+1}) = \frac{\beta_{n+1}^{\alpha_{n+1}}}{(\alpha_{n+1}-1)!} \theta^{\alpha_{n+1}-1} e^{-\beta_{n+1} \theta} \end{align*} By the law of total probabilities, \begin{align*} \int_0^\infty {h(\theta | x_1, \dots, x_{n+1})}{d\theta} &= 1 \\ \iff \int_0^\infty{\theta^{\alpha_{n+1}-1} e^{-\beta_{n+1} \theta}}{d\theta} &= \frac{(\alpha_{n+1}-1)!}{\beta_{n+1}^{\alpha_{n+1}}} \\\\ \implies f(x_{n+1} | x_1, \dots, x_n) &= x_{n+1} \frac{\beta_n^{\alpha_n}}{(\alpha_n-1)!} \frac{(\alpha_{n+1}-1)!}{\beta_{n+1}^{\alpha_{n+1}}} \\&= \frac{(\alpha_{n+1}-1)!}{(\alpha_n-1)!} \frac{\beta_n^{\alpha_n}}{\beta_{n+1}^{\alpha_{n+1}}} x_{n+1} \\&= \frac{(2n+3)!}{(2n+1)!} \frac{(\lambda + n\bar{x}_n)^{2(n+1)}}{(\lambda + (n+1) \bar x_{n+1})^{2(n+2)}} x_{n+1} \\&= (2n+2)(2n+3) \left( \frac{\lambda + n\bar{x}_n}{\lambda + (n+1) \bar x_{n+1}} \right)^{2(n+1)} \frac{x_{n+1}}{(\lambda + (n+1) \bar x_{n+1})^2} \\&= \frac{(2n+2)(2n+3) x_{n+1}}{(\lambda + (n+1) \bar x_{n+1})^2} \left( 1 - \frac{x_{n+1}}{\lambda + (n+1) \bar x_{n+1}} \right)^{2(n+1)} \end{align*}