# Posterior Distribution of a Prior Variable

Let $X_{1},\dots,X_{n}$ be a random sample from an exponential distribution with density $f(x;\theta)=\theta e^{-\theta x}$, $x>0$ (having mean $1/\theta$). Assume a prior density for $\theta$ which is also exponential with mean $1/\beta$, where $\beta$ is known. Prove that the posterior distribution of $\beta$ is a gamma distribution.

--In general, the posterior distribution would be $\pi(\beta\mid \mathbf{x})=f(\mathbf{x}\mid \beta)\pi(\beta)/m(\mathbf{x})$, where $m(\mathbf{x})$ is the marginal probability of $x$. The part that I am stuck at has to do with being given $f(\mathbf{x}\mid\theta)$, not $f(\mathbf{x}\mid\beta)$. Any help is appreciated!

• If $\beta$ is assumed to be known, then a posteriori it is the same --- it is still known, and has the same value as before. Are you sure it's not the posterior distribution of $\theta$ that you're looking for? – Kirill Jul 13 '13 at 20:53
• It would make more sense to me, but it is the posterior distribution of $\beta$ that I need. It is from a past PhD qualifying exam. – Kirk Fogg Jul 13 '13 at 20:54
• Are you definitely certain it's not a typo? If $\beta$ is known with certainty a priori, it is still known with certainty a posteriori. – Kirill Jul 13 '13 at 20:55
• Because the posterior over $\theta$ is $P(\theta|x) \propto \theta^n e^{-\theta(\beta+\sum_i x_i)}$, and, in fact, is a gamma distribution. – Kirill Jul 13 '13 at 20:58
• Perhaps this is what the question was meant to ask. It seems much more straight forward this way. – Kirk Fogg Jul 13 '13 at 21:03

## 2 Answers

The likelihood function is $$L(\theta) = \prod_{i=1}^n \left(\theta e^{-\theta x_i} \right) = \theta^n e^{-\theta(x_1+\cdots+x_n)}\text{ for }\theta>0.$$

The prior density is $$f(\theta) = \frac1\beta e^{-\theta/\beta}\text{ for }\theta>0.$$

Multiply the two: $$\text{constant}\cdot \theta^n e^{-\theta\left(x_1+\cdots+x_n+\frac1\beta\right)},$$ where in this case "constant" means not depending on $\theta$.

Suitably normalized, that is a gamma density. The normalizing constant will generally differ from the "constant" above. You can find the normalizing constant just by looking at tabulated facts about the gamma density.

Possibly you don't even need the normalizing constant. If one parametrizes the family of gamma densities as $$f(t) = \text{constant}\cdot t^{\alpha-1} e^{-\gamma t}\text{ for }t>0,$$ then in this case we have $\alpha = n+1$ and $\gamma=x_1+\cdots+x_n+\frac1\beta$.

By Bayes theorem, the posterior distribution over $\theta$ is given by $$P(\theta\mid x_1,\ldots,x_n) = \frac{P(x_1,\ldots,x_n\mid\theta)P(\theta)}{P(x_1,\ldots,x_n)}.$$ Now, since $x_i$ are all independent, we have $$P(x_1,\ldots,x_n\mid\theta) = P(x_1\mid\theta)\cdots P(x_n\mid\theta) = \theta^n e^{-\theta\sum_i x_i}.$$ Also, the prior is $P(\theta) = \beta e^{-\beta\theta}$. Because $P(x_1,\ldots,x_n)$ is not a function of $\theta$, we can write $$P(\theta\mid x_1,\ldots,x_n) = \text{const}\times\theta^n e^{-\theta\sum_i x_i-\theta\beta},$$ where the constant of proportionality will be fixed by requiring that $\int P(\theta\mid x_1,\ldots,x_n)\,d\theta=1$.