# posterior density for bayesian estimations

Suppose that the number of accidents occurring daily in a certain plant has a Poisson distribution with an unknown mean $\lambda$. Based on previous experience in similar industrial plants, suppose that a statistician's initial feeling about the that possible value of $\lambda$ can be expressed by an exponential distribution with parameter 2. What are the two Bayes estimates of $\lambda$?

I tried to find the posterior density, but I got stuck at: $$f(\theta|X_1,...,X_n) = \frac{k}{\int_0^\infty k d\theta}$$ where $k = e^{-(n+2)\theta}\theta^{X_1+...+X_n}$. I'm not sure if I did this right and if I did, I don't know how to integrate the denominator of that fraction.

-

Let $\lambda_0$ be the true rate at which the accidents occur. Let ${\mathbf X}=\{X_1, X_2, \ldots\}$ be the number of accidents on day 1, day 2, etc. Then the posterior distribution is given by Bayes' theorem $$p(\lambda|\mathbf{X}) = \frac{p(\mathbf{X}|\lambda)p(\lambda)}{p(\mathbf{X})}\propto p(\mathbf{X}|\lambda)p(\lambda)$$

The prior is an exponential distribution with rate $\phi = 2$ such that $$p(\lambda) = \phi \exp(-\phi\lambda)$$

We furthermore assume that the numbers of accidents per day are independent (this is probably not quite true). Consequently, we may write $$p(\mathbf{X}|\lambda) = \prod_{i=1}^N p(X_i|\lambda),$$ where $p(X_i|\lambda)$ is the probability that $X_i$ accidents occur on day $i$ given an accident rate $\lambda$. Assuming a Poisson distribution we have $$p(X_i|\lambda)=\frac{\lambda^{X_i}}{X_i}\exp(-\lambda).$$

Plugging into the product in the equation above we find $$p(\mathbf{X}|\lambda) = \exp(-N\lambda) \frac{\lambda^{X_1+X_2+\ldots}}{X_1!X_2!\ldots}$$

The posterior is thus proportional to $$p(\lambda|\mathbf{X})\propto \exp(-N\lambda) \frac{\lambda^{X_1+X_2+\ldots}}{X_1!X_2!\ldots}\times \phi \exp(-\phi\lambda)$$ and we can find estimates of $\lambda_0$ by e.g. finding the maximum of the posterior.

In particular, you asked for the mean and median. Note that the median is not the same as the maximum. You can read up on the topics here: Mean, Median, Mode (Maximum)

We define a normalized posterior distribution $$p'(\lambda)\equiv A p(\lambda|\mathbf{X}),$$ where $A$ is chosen such that $\int_0^\infty d\lambda \, p'(\lambda) = 1$. I will leave it for you to carry out the integration.

Then the median $m$ is given by $$\frac{1}{2}=\int_0^m d\lambda \, p'(\lambda).$$

The mean $\mu$ is given by $$\mu = \int_0^\infty d\lambda \, p'(\lambda)\lambda.$$

Edit 1: Corrected a factor of $N$ that was missing in the expression for the conditional probability $p(\mathbf{X}|\lambda)$.

Edit 2: Added methods to estimate the mean and median of the posterior.

-
going from $p(X_i|\lambda)$ to $p(X|\lambda)$, shouldn't $e^{-\lambda}$ be multiplied N times? –  woaini Nov 30 '12 at 17:44
also, if you were to estimate $\lambda$ from the posterior, don't you need the complete posterior density? –  woaini Nov 30 '12 at 17:46
Yes, the factor of N was missing. Corrected. However, you do not need the full posterior distribution if you use appropriate estimators (e.g. maximize the posterior). You can find more information on page 3 of this publication: arxiv.org/abs/0906.0664 However, the full posterior is needed if you compare different models with one another. –  Till Hoffmann Nov 30 '12 at 17:47
I have to find the estimate using the mode (which I think is through maximizing the posterior) and the mean of the posterior density. How do I find the mean of the posterior distribution without the full thing? –  woaini Nov 30 '12 at 18:02
The answer is updated to show how to determine the mean and median of the posterior. –  Till Hoffmann Nov 30 '12 at 19:18