0
$\begingroup$

Let $\beta$ be a $p\times 1$ vector, $Y$ a $p\times 1$ vector, and $X$ a $n\times p$ matrix. Suppose we have the following density functions $$f(\beta) = \frac{1}{(2\pi)^{p/2} \lambda^{-p}} \exp\left(-\frac{\lambda}{2} \beta^T \beta \right)$$ $$f(Y\mid \beta ) =\frac{1}{(2\pi)^{n/2}} \exp\left(-\frac{1}{2}(Y-X\beta)^T(Y-X\beta)\right)$$

How can I find $f(Y) = f(Y\mid \beta) f(\beta)$? By definition we have

$$f(Y) = \int_{\Theta} f(Y\mid \beta) f(\beta) \,d\beta $$

But this $p$-dimensional integral seems difficult to evaluate.

$\endgroup$
1
  • $\begingroup$ Not really. For given $Y$, complete the square (in $\beta$) in the exponent of $f(Y|\beta)f(\beta)$ and ... $\endgroup$ Nov 13, 2017 at 22:59

1 Answer 1

0
$\begingroup$

For conjugate families, you rarely have to compute $f_{Y}(y)$. Note that \begin{align*} f(\beta|y) &\propto f(\beta)f(y|\beta) \\ &\propto \exp(-2^{-1}\lambda \beta^{T}\beta)\exp(-2^{-1}(y-x\beta)^{T}(y-x\beta)) \\ &= \exp(-2^{-1}(\lambda \beta^{T}\beta + y^{T}y -2y^{T}x\beta+(x\beta)^{T}x\beta)) \\ &\propto \exp(2^{-1}(\lambda\beta^{T}\beta-2y^{T}x\beta + \beta^{T}(x^{T}x)\beta)) \\ &= \exp(2^{-1}(\beta^{T}(\lambda I + x^{T}x)\beta -2y^{T}x\beta)) \end{align*} Also note that, \begin{align*} \exp(-2^{-1}(\beta-\mu)^{T}\Sigma^{-1}(\beta-\mu)) &= \exp(2^{-1}(\beta^{T}\Sigma^{-1}\beta-2\mu^{T}\Sigma^{-1}\beta+\mu^{T}\Sigma^{-1}\mu)) \\ &\propto \exp(2^{-1}(\beta^{T}\Sigma^{-1}\beta-2\mu^{T}\Sigma^{-1}\beta)) \end{align*} Therefore, by taking $(\lambda I+x^{t}x)=\Sigma^{-1}$ and $y^{T}x = \mu\Sigma^{-1}$, conclude that \begin{align*} f(\beta|y) &\propto \exp(2^{-1}(\beta^{T}\Sigma^{-1}\beta-2\mu^{T}\Sigma^{-1}\beta)) \end{align*} That is, $\beta|Y \sim N(\mu,\Sigma)$. From the definitions, obtain that $\Sigma=(\lambda I + x^{t}x)^{-1}$ and $\mu=y^{t}x(\lambda I + x^{t}x)^{-1}$. In case you are also after $f(y)$, note that \begin{align*} f(\beta|y) &= \frac{f(\beta)f(y|\beta)}{f(y)} \\ f(y) &= \frac{f(\beta)f(y|\beta)}{f(\beta|y)} \end{align*} Since we obtained $f(\beta|y)$, and $f(\beta)$ and $f(\beta|y)$ are given, you obtain $f(y)$.

$\endgroup$
2
  • $\begingroup$ So what is $f_Y (y)$? $\endgroup$
    – 3x89g2
    Nov 14, 2017 at 18:04
  • $\begingroup$ You actually want $f_{Y}(y)$? The title of the questions says you are after the posterior. In any case, I added $f(y)$ to the answer. $\endgroup$
    – madprob
    Nov 14, 2017 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.