I am trying to solve this problem from a past exam paper:

Let $X_1, \dots, X_n$ be a random sample of size $n$ from the density:

$p(x; \theta) = exp \{ -(x-\theta) \}, \qquad x > \theta$

In a Bayesian framework, assume that the prior distribution for $\theta$, $\pi(\theta)$, is a Uniform distribution on the interval $(0, B)$, where $B$ is a known positive constant. Determine the posterior distribution of $\theta$, clearly stating the normalising constant.

I know that posterior $\propto$ prior $\times$ likelihood.

Prior: $$\pi(\theta) = \dfrac{1}{B}$$

Likelihood: $$p(\underline{x} ; \theta) = \prod_{i=i}^{n} exp \{ - (x_i - \theta) \} = exp \left\{ - \left(\sum_{i=1}^{n} x_i - n\theta\right) \right\} $$

EDIT: Changed equality to proportionality in response to the comment below.

Posterior: $$ \pi(\theta \mid \underline{x}) \propto \dfrac{1}{B} \times exp \left\{ - \left(\sum_{i=1}^{n} x_i - n\theta\right) \right\} $$

Now I am stuck. I know the posterior distribution needs to integrate to 1, and this is the way I can find the normalising constant. However, I am unsure about how to find the limits of the integration and how to actually compute this integration (I assume the integration is with respect to $\theta$?)


1 Answer 1


Minor comment: posterior is proportional to the product of prior and maximum likelihood, not equal.

The integration is indeed with respect to $\theta$. To find the limits refer to the definition of p.d.f., namely it's non-zero when $\theta < x$. Hence, $$ \pi(\theta \mid \underline{x}) = \frac 1 {\mathcal{Z}} \cdot \dfrac{1}{B} \times \exp \left\{ - \left(\sum_{i=1}^{n} x_i - n\theta\right) \right\} $$ and we need to find the constant $\mathcal{Z}$. Integrating one gets that $$ \int \pi(\theta \mid \underline{x}) ~d\theta = \frac 1 {\mathcal{Z}} \cdot \frac 1 {nB} e^{-\sum_{i=1}^n x_i} \int_{-\infty}^{\max(x_i)} e^{n\theta} ~ d(n\theta) = \frac 1{nB} {e^{n\max(x_i) - \sum_{i=1}^n x_i}} \cdot \frac 1 {\mathcal{Z}} = 1 \implies \\ \mathcal{Z} = Bn \cdot e^{- n\max(x_i) + \sum_{i=1}^n x_i}. $$ Finally, the posterior distribution can be written as follows $$ \pi(\theta \mid \underline{x}) = \frac 1 {B^2n} \cdot \exp\left\{n[\max(x_i) + \theta] -2 \sum_{i=1}^n x_i\right\}. $$

  • $\begingroup$ Thanks for your minor comment - I have updated the post accordingly. Thanks also for explaining where the limits of the integration come from, that makes sense. Where do the extra $n$s come from e.g. in $\dfrac{1}{nB}$ and $d(n \theta)$? Is that just because we have $e^{n\theta}$ in the integral? $\endgroup$
    – meenaparam
    Aug 15, 2018 at 12:27
  • $\begingroup$ I multiply and divide by $n$, in order to make the integral look more friendly ;) $\endgroup$
    – tortue
    Aug 15, 2018 at 12:33
  • $\begingroup$ Ah ok, thanks for clarifying and for your help with this! $\endgroup$
    – meenaparam
    Aug 15, 2018 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.