Find posterior distribution of Gaussian likelihood with Gaussian prior

Question

Let $\beta$ be a $p\times 1$ vector, $Y$ a $p\times 1$ vector, and $X$ a $n\times p$ matrix. Suppose we have the following density functions $$f(\beta) = \frac{1}{(2\pi)^{p/2} \lambda^{-p}} \exp\left(-\frac{\lambda}{2} \beta^T \beta \right)$$ $$f(Y\mid \beta ) =\frac{1}{(2\pi)^{n/2}} \exp\left(-\frac{1}{2}(Y-X\beta)^T(Y-X\beta)\right)$$

How can I find $f(Y) = f(Y\mid \beta) f(\beta)$? By definition we have

$$f(Y) = \int_{\Theta} f(Y\mid \beta) f(\beta) \,d\beta $$

But this $p$-dimensional integral seems difficult to evaluate.

Answer 1

For conjugate families, you rarely have to compute $f_{Y}(y)$. Note that \begin{align*} f(\beta|y) &\propto f(\beta)f(y|\beta) \\ &\propto \exp(-2^{-1}\lambda \beta^{T}\beta)\exp(-2^{-1}(y-x\beta)^{T}(y-x\beta)) \\ &= \exp(-2^{-1}(\lambda \beta^{T}\beta + y^{T}y -2y^{T}x\beta+(x\beta)^{T}x\beta)) \\ &\propto \exp(2^{-1}(\lambda\beta^{T}\beta-2y^{T}x\beta + \beta^{T}(x^{T}x)\beta)) \\ &= \exp(2^{-1}(\beta^{T}(\lambda I + x^{T}x)\beta -2y^{T}x\beta)) \end{align*} Also note that, \begin{align*} \exp(-2^{-1}(\beta-\mu)^{T}\Sigma^{-1}(\beta-\mu)) &= \exp(2^{-1}(\beta^{T}\Sigma^{-1}\beta-2\mu^{T}\Sigma^{-1}\beta+\mu^{T}\Sigma^{-1}\mu)) \\ &\propto \exp(2^{-1}(\beta^{T}\Sigma^{-1}\beta-2\mu^{T}\Sigma^{-1}\beta)) \end{align*} Therefore, by taking $(\lambda I+x^{t}x)=\Sigma^{-1}$ and $y^{T}x = \mu\Sigma^{-1}$, conclude that \begin{align*} f(\beta|y) &\propto \exp(2^{-1}(\beta^{T}\Sigma^{-1}\beta-2\mu^{T}\Sigma^{-1}\beta)) \end{align*} That is, $\beta|Y \sim N(\mu,\Sigma)$. From the definitions, obtain that $\Sigma=(\lambda I + x^{t}x)^{-1}$ and $\mu=y^{t}x(\lambda I + x^{t}x)^{-1}$. In case you are also after $f(y)$, note that \begin{align*} f(\beta|y) &= \frac{f(\beta)f(y|\beta)}{f(y)} \\ f(y) &= \frac{f(\beta)f(y|\beta)}{f(\beta|y)} \end{align*} Since we obtained $f(\beta|y)$, and $f(\beta)$ and $f(\beta|y)$ are given, you obtain $f(y)$.