# Finding the likelihood function and Posterior Distribution

Let $$\underline{X}=(X_1,...X_n)$$ be an i.i.d. random sample from an exponential distribution, with probability density function given by

$$f(x; \theta)=\lambda exp$${$$-\lambda x$$} , x>0 where $$\lambda$$ is an unknown parameter taking values in $$\mathbb{R^+}$$

A Derive the likelihood function $$L(\lambda; \underline{X})$$ and derive the Fisher information $$I(\lambda)$$ measuring the amount of information that $$\underline{X}$$ carries about $$\lambda$$

B Given the Jeffreys prior for $$\lambda$$ is $$\pi_J(\lambda)\propto \lambda^{-1}$$, derive the posterior distribution for $$\lambda$$. Find the mean and variance of $$\lambda| \underline{X}$$

I know that for A I need to use the product up to n and then for the fisher info it is $$\frac{1}{-E(l''(\lambda)}$$ but I'm not sure how to do this for my given model.

• As you say, for the likelihood function, you can take the product, which would be $\prod_i \lambda \exp(-\lambda x_i) = \lambda^n \exp(-\lambda \sum_i x_i)$ – Henry Dec 10 '20 at 9:43
• I have managed to work out A do you have any guidance as to work out the posterior distribution? @Henry – user847304 Dec 10 '20 at 10:01
• If the prior for $\lambda$ is $\pi(\lambda)$ then the posterior is $\pi(\lambda \mid x) = \dfrac{\pi(\lambda) \,f(x;\lambda)}{\int_{\lambda'} \pi(\lambda') \,f(x;\lambda')\, d\lambda'}$ – Henry Dec 10 '20 at 10:04
• @B24Panther : the denominator showed by Henry is correct but you can avoid to calculate it. When multiplying prior and likelihood you immediately recognize the kernel of a known density... – tommik Dec 10 '20 at 10:12

## 1 Answer

A: the likelihood is the following:

$$L(\lambda)=\lambda^n e^{-\lambda\Sigma_i X_i}$$

To calculate Fischer information, as you stated, you have to calculate

$$-n\mathbb{E}\Bigg[\frac{\partial^2}{\partial\lambda^2}\log f(x;\lambda)\Bigg]$$

thus simply:

$$\log f=\log\lambda-\lambda x$$

$$\frac{\partial^2}{\partial\lambda^2}\log f=-\frac{1}{\lambda^2}$$

So the Fischer information of the n-tuple $$(X_1\dots,X_n)$$ is

$$\frac{n}{\lambda^2}$$

B

The posterior is the following

$$\pi(\lambda|\mathbf{x})\propto \pi(\lambda)\cdot p(\mathbf{x}|\lambda)$$

that is

$$\pi(\lambda|\mathbf{x})\propto \lambda^{-1}\times \lambda^n e^{-\lambda\Sigma_i X_i}= \lambda^{n-1}e^{-\lambda\Sigma_i X_i}$$

We immediately recongize the kernel of a $$Gamma(n;\Sigma_i X_i)$$ thus

$$\mathbb{E}[\lambda|\mathbf{x}]=\frac{n}{\Sigma_i X_i}$$

and

$$\mathbb{V}[\lambda|\mathbf{x}]=\frac{n}{(\Sigma_i X_i)^2}$$

• see for $\pi(\lambda)$ would I not sub in the jefreys prior given? – user847304 Dec 10 '20 at 10:30
• @B24Panther ?? just multiply the prior and the likelihood...I added an intermediate passage..hope it's clear now – tommik Dec 10 '20 at 10:36
• Thanks for your help :) – user847304 Dec 10 '20 at 11:04