# Bayes Estimator of normal distribution and normal prior

I am preparing for and exam and I am stuck at this question:

Let $X_1,\cdots,X_n \sim N(\theta,1)$ with parameter $\theta$. The prior information $\pi$ on $\theta$ is given by an $N(0,\tau^2)$ distribution.

I have to calculate the posteriori distribution on $\theta$ and the Bayes estimator.

I began as follows:

First calculate $$\pi(\theta)p_{\theta}(X) = \frac 1 {\tau \sqrt{2 \pi}} \exp \left ( - \frac 1 2\left ( \frac \theta \tau\right ) ^2 \right) \cdot \prod_{i = 1}^n \frac 1 {\sqrt{2 \pi}} \exp \left ( - \frac 1 2 (x_i - \theta)^2 \right)$$ which is $$\frac 1 {\tau \sqrt{2 \pi}^{n+1}}\cdot \exp \left ( - \frac 1 2 \left( \frac \theta \tau \right)^2 \right ) \exp \left ( \sum_{i =1}^n - \frac 1 2 (x_i - \theta)^2 \right)$$

But what now ?

-
Isn't the $\sigma$ supposed to be $\tau$? – Johnny Westerling Mar 26 '13 at 10:52
Yes. Corrected :) – Andre Mar 26 '13 at 10:55

I am not going to write out every step, but it is rather long and I am sick of typing latex.

If you scroll down and should notice normal is conjugate prior to itself and it actually gives you the answer there. Your example is the second one with $\mu_0 = 0$

As a general tip, when doing this type of questions, you should drop the $\frac{1}{\sqrt{2\pi}}$, since your expression is only up to a constant of proportionality anyway. (**)

You need to expand your expression and write all the exponentials term together, then factorise it as $-\frac{(\theta-y)^2}{2z}$ for some expression $y$. where $y$ and $z$ will be in term of $\tau$. Then you observe, this is proportional the normal distribution with mean and variance given in the wikipedia article.

Bayes estimaor is given by the mean of prosterior if your loss function is $E(\theta-\hat{\theta})^2$

EDIT: Note since this is distribution for $\theta$, you can just drop every multiplicative which is not a function of $\theta$, so even if you have terms like $\frac{1}{\sqrt{\tau}}$. These can be safely ignored in the step (**)

-
Thanks for that. Maybe you cant help a bit with the calculation: I got now that $\pi(\theta) p_\theta(x) \sim \exp(-\frac 1 2 \left (\left ( \frac \theta \tau \right)^2 + \sum_{i = 1}^n (x_i - \theta)^2 \right ))$. So I want to transform $\left ( \frac \theta \tau \right)^2 + \sum_{i = 1}^n (x_i - \theta)^2)$ into something like $\left ( \frac {\theta - y} z \right)^2$ right ? – Andre Mar 26 '13 at 12:07
@André yes, that is right. You expand the $(x_i-\theta)^2$ and muptiply the top and bottom by $\tau^2$. so you have lots of $\dfrac{[(n\tau^2+1)\theta^2-2(\sum\limits_{i=1}^n x_i)\theta+...]}{\tau^2}$ where the ... can be ignored becase it is just a constant. Now you divide it by $(n\tau^2+1)$ on top and bottom and complete the square to $(\frac{\theta-y}{z})^2$ form – Lost1 Mar 26 '13 at 12:49
Oh yes. I forgot the thing about the constants again :D Great help ! – Andre Mar 26 '13 at 12:51
@André the key to this kind of conjugate prior questions is that you drop things which are not variables in the prosterior distribution. This can includes the DATA and the PARAMETERS of POSTERIOR. As a future tip, any type of standard distribution being used as priors, it is probably on the wikipedia page I gave you. It is a good place to check your answers. – Lost1 Mar 26 '13 at 12:54

The posteriori distribution is given in your case by:

$$\pi(\theta|x)=\frac{p_{\theta}(X)\pi(\theta)}{\int_{\Omega}p_{\theta}(X)\pi(\theta)\,d\theta}.$$

I'll let you make the calculations now yourself; to check - here's what I believe is a generalized solution. For $X_{1},X_{2},...,X_{n}$ iid $\mathcal{N}(\theta,\sigma^2)$, and a priori distribution $\theta\sim\mathcal{N}(\mu,\tau^2)$, you should obtain posteriori distribution $\mathcal{N}(\mu_{\ast},\tau^2_{\ast})$, where:

$$\mu_{\ast}=\frac{\frac{n}{\sigma^2}\bar{x}+\frac{\mu}{\tau^2}}{\frac{n}{\sigma^2}+\frac{1}{\tau^2}}\quad\text{and}\quad\tau^{2}_{\ast}=\left(\frac{n}{\sigma^2}+\frac{1}{\tau^2}\right)^{-1}$$

As for the Bayesian estimator - well, I believe that that would depend on your risk function; with a MSE function, you should obtain $\theta^{B}_{\Pi}=\mu_{\ast}$.

-