# Posterior for normal likelihood, normal prior

We are given the following information: $$\Theta = \mathbb{R}, Y \in \mathbb{R}, p_\theta=N(\theta, 1), \pi = N(0, \tau^2)$$. I am asked to compute the posterior. So I know this can be computed with the following 'adaptation' of Bayes's Rule: $$\pi(\theta \mid Y) \propto p_\theta(Y)\pi(\theta)$$. Also, I've used that we have a normal distribution for the likelihood and a normal distribution for the prior, and we know that the posterior is going to be a normal distribution again due to conjugacy. But I don't really know how to derive the parameters of the posterior normal distribution. So far, I have computed the following:

$$\pi(\theta \mid Y) \propto p_\theta(Y)\pi(\theta)$$ $$= \exp(\theta/\tau^2+(Y-\theta)^2/2)$$ $$=\exp((2\theta^2+\tau^2(Y-2\theta Y+\theta^2))/\tau^2)$$ $$\propto \exp(\theta^2/\tau^2-\theta/Y+\theta^2/2)$$

But now I'm stuck. I know we need to work towards a form $$\exp((\theta-a)^2/b^2)$$, but I don't know how to get there. Any tips or a derivation of the parameters would be appreciated.

(this question was also asked here, Gaussian product - posterior probability distribution but it was never answered and less detailed.)

## 1 Answer

Your notation is confusing because it is not clear how the hierarchical model is specified. Based on the expression $$\pi(\theta \mid Y) \propto p_\theta(Y)\pi(\theta)$$, I can only assume that the model should be $$Y \mid \Theta \sim \operatorname{Normal}(\mu = \Theta, \sigma^2 = 1), \\ \Theta \sim \operatorname{Normal}(\mu = 0, \sigma^2 = \tau^2).$$

As the posterior for $$\Theta$$ is proportional to the joint density, we write \begin{align*} \pi_{\Theta \mid Y}(\theta \mid y) &\propto f_{Y \mid \Theta}(y_i \mid \theta)\pi_\Theta(\theta) \\ &\propto \exp\left(-\frac{(y - \theta)^2}{2}\right)\exp\left(-\frac{\theta^2}{2\tau^2}\right)\\ &= \exp\left(-\frac{\tau^2 (y-\theta)^2 + \theta^2}{2\tau^2}\right). \end{align*} This motivates us to complete the square with respect to $$\theta$$:
\begin{align*} \theta^2 + \tau^2(y-\theta)^2 &= (1+\tau^2)\theta^2 - 2\tau^2 y \theta + \tau^2 y^2 \\ &= (1+\tau^2) \left(\theta^2 - \frac{2\tau^2 y}{1+\tau^2} \theta\right) + (\tau y)^2 \\ &= (1+\tau^2) \left(\theta - \frac{\tau^2 y}{1+\tau^2}\right)^{\!2} - \frac{(\tau^2 y)^2}{1+\tau^2} + (\tau y)^2 \\ &= (1 + \tau^2)\left(\theta - \frac{\tau^2 y}{1+\tau^2} \right)^{\!2} + \frac{(\tau y)^2}{1+\tau^2}. \end{align*} Consequently, $$\pi_{\Theta \mid Y}(\theta \mid y) \propto \exp \left( - \frac{(\theta - \kappa)^2}{2\varsigma^2} \right) \exp\left(-\frac{y^2}{2(1+\tau^2)}\right) \propto \exp \left( - \frac{(\theta - \kappa)^2}{2\varsigma^2} \right)$$ where $$\kappa = \frac{\tau^2 y}{1+\tau^2}, \quad \varsigma^2 = \frac{\tau^2}{1+\tau^2}$$ represent the posterior mean and variance of $$\Theta$$. The constant factor that was dropped in the last proportionality is independent of $$\theta$$. Therefore, our posterior is $$\Theta \mid Y \sim \operatorname{Normal}(\mu = \kappa, \sigma^2 = \varsigma^2).$$ Since $$\tau$$ is typically used to denote precision rather than variance, i.e. $$\tau \sigma^2 = 1$$, if this is the intended parametrization, the posterior mean and variance would be written as $$\kappa = \frac{y}{1+\tau}, \quad \varsigma^2 = \frac{1}{1+\tau}.$$