Exercise: Assume a Bayesian setup in which $X_1,\ldots,X_n\mid\Theta\stackrel{iid}{\sim}N(\theta,\sigma^2)$ and $\Theta$ gets assigned a prior distribution. Take a flat prior on $\Theta: f_\Theta(\theta)\propto 1$. We consider testing $H_0:\theta\leq \theta_0$ versus $H_1:\theta > \theta_0$. Show that the posterior probability of $H_0$ equals $$\Phi\bigg(\sqrt{n}\dfrac{\theta_o -\bar{X}_n}{\sigma}\bigg)$$

My approach: I need to show that $\mathbb{P}(\theta\leq \theta_0\mid X) = \Phi\bigg(\sqrt{n}\dfrac{\theta_o -\bar{X}_n}{\sigma}\bigg)$, and I know that $f_{\Theta\mid X}(\theta\mid x)\propto f_\Theta(\theta)\,f_{X\mid\Theta}(x\mid\theta) \propto\prod\limits_{i = 1}^n\exp\bigg(\dfrac{(x_i - \theta)^2}{\sigma^2}\bigg) = \exp\bigg(-\dfrac{\sum_{i = 1}^nx_i^2}{n\sigma^2} + \dfrac{2\theta\frac{1}{n}\sum_{i = 1}^n x_i}{2\sigma^2} - \dfrac{\theta^2}{\sigma^2}\bigg)$. If I'm not mistaken, I need to find $a$ and $b$ in $\Theta|X\sim N(a,b^2)$ so that I can normalise $\Theta$. Unfortunately I can't find a way to write $\exp\bigg(-\dfrac{\sum_{i = 1}^nx_i^2}{n\sigma^2} + \dfrac{2\theta\frac{1}{n}\sum_{i = 1}^n x_i}{2\sigma^2} - \dfrac{\theta^2}{\sigma^2}\bigg)$ as $\exp\bigg(-\dfrac{(\theta - a)^2}{b^2}\bigg)$.

Question: How do I solve this exercise/what am I missing?

Thanks in advance!


The posterior distribution of $\theta$ (with $\sigma$ fixed) is proportional to the likelihood, which is given by $$\mathcal L(\theta \mid \boldsymbol x) \propto \exp \left( - \frac{1}{2\sigma^2} \sum_{i=1}^n (x_i - \theta)^2 \right).$$ We may ignore the improper prior $\pi(\theta) = 1$. We can ignore all other factors that do not depend on $\theta$. Now if we let $\bar x = n^{-1} \sum_{i=1}^n x_i$ be the sample mean, we have $$\begin{align*} \sum_{i=1}^n (x_i - \theta)^2 &= \sum_{i=1}^n (x_i - \bar x + \bar x - \theta)^2 \\ &= \sum_{i=1}^n (x_i - \bar x)^2 + 2(x_i - \bar x)(\bar x - \theta) + (\bar x - \theta)^2 \\ &= n(\bar x - \theta)^2 + 2(\bar x - \theta) \sum_{i=1}^n (x_i - \bar x) + \sum_{i=1}^n (x_i - \bar x)^2 \\ &= n(\bar x - \theta)^2 + 0 + \sum_{i=1}^n (x_i - \bar x)^2, \end{align*}$$ where the second term is zero because the sample total equals $n$ times the sample mean. Hence $$f(\theta \mid \boldsymbol x) \propto \exp\left( - \frac{n(\bar x - \theta)^2}{2\sigma^2}\right)\exp\left(-\frac{1}{2\sigma^2} \sum_{i=1}^n (x_i - \bar x)^2\right),$$ and as this second exponential factor is independent of $\theta$, it too is just a constant of proportionality with respect to the posterior distribution of $\theta$. We conclude that the posterior density of $\theta$ is normal, with mean $\bar x$ and variance $\sigma^2/n$; thus the posterior probability of $H_0$ is $$\Pr[H_0 \mid \boldsymbol x] = \Pr[\theta \le \theta_0 \mid \boldsymbol x] = \Pr\left[\frac{\theta - \bar x}{\sigma/\sqrt{n}} \le \frac{\theta_0 - \bar x}{\sigma/\sqrt{n}} \mid \boldsymbol x \right] = \Pr\left[Z \le \frac{\theta_0 - \bar x}{\sigma/\sqrt{n}}\right] = \Phi\left(\sqrt{n}\frac{\theta_0 - \bar x}{\sigma}\right).$$

  • $\begingroup$ Thanks for your reply! There is just one thing that I don't understand! Why is the mean of the posterior density of $\theta$ equal to $\bar{x}$? We have $f(\theta\mid x)\propto \exp\bigg(-\dfrac{n(\bar{x} - \theta)^2}{2\sigma^2}\bigg)$. If $\bar{x}$ was the mean of the posterior density wouldn't we have $f(\theta\mid x)\propto \exp\bigg(-\dfrac{n(\theta - \bar{x})^2}{2\sigma^2}\bigg)$? Could you just explain this final step? Thanks! $\endgroup$ – titusAdam Jan 24 '18 at 10:29
  • $\begingroup$ $(a-b)^2 = (b-a)^2$. $\endgroup$ – heropup Jan 24 '18 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.