2
$\begingroup$

I have the prior density function:

$$e^{-\theta} \text{ for } \theta > 0$$

and the likelihood function: $e^{\theta -x}$ for $x \geq \theta$

I have gotten the following in my attempt to derive the posterior distribution:

$$L(\theta) = \prod e^{\theta - x_i} = e^{n\theta}e^{-\sum x_i} \mathbb{I}_{\min X_i}$$

$$\pi(\theta \mid x) = \frac{ e^{-\sum x_i} e^{\theta(n-1)} \mathbb{I}_{\min X_i}}{e^{-\sum x_i}\int_0^{\min(x)}e^{n\theta}e^{-\theta} \, d\theta}$$

$$= \frac{e^{\theta(n-1)}\mathbb{I_{\min X_i}}}{\frac{1}{n-1}e^{\min(x)(n-1)} - \frac{1}{n-1}}$$

Is this correct? Does this further simplify, or is the posterior simply non-standard?

$\endgroup$
2
$\begingroup$

and the likelihood function: $e^{\theta-x}$ for $x≥θ$

First, this is not the likelihood but the Model Density.

You are almost there but I suggest you to approach Bayesian issue in a different way: do not worry about the posterior denominator... it is wasted time!

  • Prior:

$$\pi(\theta)=e^{-\theta}$$

$\theta>0$

  • Likelihood:

$$p(\mathbf{x}\mid\theta) = e^{-\sum_i x_i}\cdot e^{n \theta} \cdot \mathbb{1}_{[\theta;+\infty)}(x_{(1)})$$

  • Posterior (any element not depending on $\theta$ can be trown away, it will be part of the normalization constant)

$$\pi(\theta|\mathbf{x})=C\times p(\mathbf{x}\mid\theta)\times \pi(\theta)=C \times e^{\theta(n-1)}\cdot\mathbb{1}_{(0;x_{(1)}]}(\theta)$$

Where

$$C^{-1}=\int_0^{x_{(1)}}e^{\theta(n-1)}d \theta=\frac{e^{x_{(1)}(n-1)}-1}{n-1}$$

Observe that: When you deal with the posterior $x_1,x_2,...,x_n$ are only data, not anymore rv's


A real world example using this Statistical Model

According to the thinking of Cthulhu Cult's followers, adoring the "Great Ancients" (an ancient civilization come from the stars that lived on Earth before Homo Sapiens appeared) will grant a very long life to them. In particular, story goes about a group of followers dead at the age of

$$\{X_1=518;X_2=696;X_3=230;X_4=410;X_5=821\}$$

years old

Using these observations, Prior and Model specified above, derivate the posterior density of Cult Followers' lifetime and, further request with respect to your exercise, Verify the Hypothesis that the lifetime of the Cthulhu Cult's Members is greater than 200 years

$$\pi(\theta|\mathbf{x})=\frac{4 e^{4\theta}}{e^{920}-1}\mathbb{1}_{(0;230]}(\theta)$$

You can easy check that $\pi(\theta|\mathbf{x})$ is a nice density function. for the Hypothesis Testing, I leave it as an exercise, if this is part of your program...

$\endgroup$
2
  • $\begingroup$ Your way of stating what the likelihood is does not altogether make it clear that it's a function of $\theta$ with $x_i,\quad i=1,\ldots,n$ fixed. $\qquad$ $\endgroup$ – Michael Hardy Nov 9 '20 at 6:44
  • $\begingroup$ @MichaelHardy : edited; I updated my answer with a "real World" example $\endgroup$ – tommik Nov 9 '20 at 10:10
2
$\begingroup$

I will illustrate how working with the kernel with respect to $\theta$ is easier.

For a sample $\boldsymbol x = (x_1, x_2, \ldots, x_n)$ drawn from a distribution for which $X - \theta \mid \theta \sim \operatorname{Exponential}(1)$, i.e., $X \mid \theta$ is a shifted exponential distribution with location shift $\theta$ and mean $1+\theta$, where $\theta \sim \operatorname{Exponential}(1)$ is itself random, we have the likelihood for $\theta$ $$\mathcal L(\theta \mid \boldsymbol x) = e^{-\theta} \mathbb 1 (\theta > 0) \prod_{i=1}^n e^{-(x_i-\theta)} \mathbb 1(x_i > \theta) = e^{-\theta} e^{-n \bar x} e^{n \theta} \mathbb 1 (x_{(1)} > \theta > 0) \propto e^{(n-1)\theta} \mathbb (0 < \theta < x_{(1)}) $$ where $\bar x$ is the sample mean.

While this is not exactly difficult to see, we could also have recognized if we observe that for a single observation, the model density is $$\mathcal L(\theta \mid x) = e^\theta e^{-x} \mathbb 1(\theta < x) \propto e^\theta \mathbb 1(\theta < x),$$ consequently the joint likelihood for the whole sample is proportional to $e^{-\theta} e^{n\theta} \mathbb 1(0 < \theta < x_{(1)}).$ Note this is a likelihood with finite support: $\theta$ cannot exceed the sample minimum $x_{(1)}$ nor can it be less than or equal to zero due to the prior being exponentially distributed.

On the interval $0 < \theta < x_{(1)}$, the posterior density is then proportional to $e^{(n-1)\theta}$ in such a way that the area under the curve equals $1$; i.e., we seek a constant $c$ such that $$c \int_{\theta=0}^{x_{(1)}} e^{(n-1)\theta} \, d\theta = 1.$$ As this leads to $$c = \frac{n-1}{e^{(n-1) x_{(1)}}-1},$$ we obtain the posterior density $$f(\theta \mid \boldsymbol x) = \frac{n-1}{e^{(n-1) x_{(1)}}-1} e^{(n-1)\theta} \mathbb 1 (0 < \theta < x_{(1)}).$$

$\endgroup$
2
$\begingroup$

One flaw is that you refer to $\mathbb{I}_{\min X_i}$ where apparently you mean $\mathbb I_{\min X_i \,\ge\,\theta}.$

I wouldn't bother with that method of computing the normalizing constant. First you conclude that the posterior density is $$ \text{constant} \times e^{\theta(n-1)} \qquad \text{for } 0<\theta<\min\{X_1,\ldots,X_n\} $$ and then evaluate the integral that tells you what the constant is. I.e. don't bother with factors like $e^{-\sum_i x_i}$ that do not depend on $\theta.$

Often it's best to leave the computation of the normalizing constant to the end, since the posterior will be recognized as having some standard form that will tell you what the normalizing constant is, and even if not, finding the necessary integral at the end will ofetn be simpler than proceeding the way you did.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.