# Bayesian Estimation with Two Parameters

Warning

After hours of trying, it has been proven (thanks, @leonbloy) that my attempt at a solution contained lots of mistakes. Maybe the correct answer is that there is no solution, but I don't know!

Homework question

Let $X_1, ..., X_n$ be a random sample from a distribution with pdf:

$$f_X(x | \lambda, \theta) = \lambda \, e^{-\lambda(x - \theta)} \, \mathbf{1}_{\{x \geq \theta \}}$$

The (independent) prior distributions of $\lambda$ and $\theta$ are:

$$\begin{cases} p(\lambda) = \frac{1}{\lambda^2} \, \mathbf{1}_{\{\lambda > 1\}} \\ p(\theta) = \frac{1}{\pi(1+\theta^2)} \, \mathbf{1}_{\{\theta \in \mathbf{R}\}} \end{cases}$$

The goal is to find Bayesian estimates under quadratic loss of $\lambda$ and $\theta$.

First approach

What we want is to find $\mathbf{E}[p(\lambda | \mathbf{x})]$ and $\mathbf{E}[p(\theta | \mathbf{x})]$.

The posterior distribution is:

$$p(\lambda, \theta \, | \, \mathbf{x}) \propto p(\theta,\lambda) f(\mathbf{x} \, | \, \lambda, \theta)$$

And then the marginal posterior distributions are:

$$\begin{cases} p(\lambda | \mathbf{x}) =\int_{-\infty}^{x_{(1)}} p(\lambda, \theta \, | \, \mathbf{x}) \, d\theta \\ p(\theta | \mathbf{x}) = \int_1^\infty p(\lambda, \theta \, | \, \mathbf{x}) \, d\lambda \end{cases}$$

Trying to answer the question

If I'm not wrong:

$$p(\lambda, \theta \, | \, \mathbf{x}) \propto \underbrace{\frac{1}{\pi\lambda^2(1+\theta)^2}}_{p(\lambda,\theta)} \, \underbrace{\lambda^n e^{-\lambda(S-n\theta)}}_{ f(\mathbf{x} \, | \, \lambda, \theta)} \qquad \left(S = \sum_{i=1}^n x_i\right)$$

I think that the normalizing constant is:

$$c_n = \frac{1}{\pi} \int_{-\infty}^{x_{(1)}} \frac{1}{1+\theta^2} \int_{1}^\infty \lambda^{n-2} e^{-\lambda(S-n\theta)} \, d\lambda \, d\theta = \frac{1}{\pi} \int_{-\infty}^{x_{(1)}}\frac{\Gamma(n-1,S-n\theta)}{(S-n\theta)^{n-1}(1+\theta^2)} \, d\theta$$

In this case $\Gamma(n-1,s-n\theta)$ is the Incomplete Gamma Function. Changing the order of integration doesn't help: $$c_n = \int_{1}^\infty \lambda^{n-2} e^{-\lambda S} \int_{-\infty}^{x_{(1)}} \frac{e^{n\theta\lambda}}{\pi(1+\theta^2)} \, d\theta \, d\lambda$$

Of course, I am also unable to find the (approximate) marginal posterior distributions.

HInt: when you compute the posterior, be careful with the domain of the parameters ($\theta$) – leonbloy Oct 31 '11 at 18:04
Aha! I forgot that $\theta \leq x$, what a mistake. Let's see if I can do something now. Thank you, @leonbloy. – Shiwen Yao Oct 31 '11 at 18:19
Sorry, I have deleted my previous comment by mistake. What I wrote was that if $\theta \leq x_1, ... , \theta \leq x_n$ then $\theta \leq \min(x_i)$, so the domain of $\theta$ is $(-\infty, x_{(1)}]$. – Shiwen Yao Oct 31 '11 at 18:43
I think the minimal sufficient statistic is $(\min\{x_1,\ldots,x_n\},\bar{x}$. So the answer should depend on $x_1,\ldots,x_n$ only through that. – Michael Hardy Nov 1 '11 at 4:24