Let\begin{align} Y\mid\beta,\sigma^2&\sim N(X\beta, \sigma^2I)\\ \beta\mid\sigma^2,\tau_1^2,\ldots,\tau_p^2&\sim N(0,\sigma^2D), \,\,D = diag(\tau_1^2, \ldots, \tau_p^2)\\ \tau_j^2\mid\lambda&\stackrel{iid}{\sim}\operatorname{Exp}(\lambda^2/2) \end{align}

and suppose that $X = I, \sigma^2 = 1$ and $\lambda = 1$. I want to show that $$E[\beta_i\mid Y_i, \tau_i] = (1-u_i)Y_i$$ with $u_i = (1+\tau_i^2)^{-1}$. I think I need to compute the posterior density of $\beta_i$ by multiplying the prior of $\beta_i$ and the likelihood of $Y_i$, but I'm having some difficulties. I think we have the following prior of $\beta_i$ and likelihood of $Y_i$: $$p(\beta_i\mid\tau_i^2) = \dfrac{1}{\sqrt{2\pi}}\exp\bigg(-\dfrac{\beta_i^2}{2\tau_i^2}\bigg), \,\,p(Y_i\mid\beta_i\sigma^2) = \dfrac{1}{\sqrt{2\pi}}\exp\bigg(-\dfrac{(Y_i -\beta_i)^2}{2}\bigg)$$

If this is correct, the posterior probability density function looks like this: $$p(\beta_i\mid Y_i,\tau_i) = \dfrac{1}{\sqrt{2\pi}}\exp\bigg(-\dfrac{\beta_i^2}{2\tau_i^2}\bigg)\dfrac{1}{\sqrt{2\pi}}\exp\bigg(-\dfrac{(Y_i -\beta_i)^2}{2}\bigg) \\= \dfrac{1}{2\pi}\exp\bigg(-\dfrac{\beta_i^2}{2\tau_i^2} - \dfrac{Y_i^2 - 2Y_i\beta_i + \beta_i^2}{2}\bigg)$$ This is the point at which I'm stuck; I don't see how the distribution corresponding to this density would have an expected value of $(1 - u_i)Y_i$...

Question: How do I solve this exercise? Am I going in the right direction?

Edit: I was wondering whether I'm allowed to just consider $$\dfrac{1}{2\pi}\exp\bigg(-\dfrac{\beta_i^2}{2\tau_i^2} - \dfrac{Y_i^2 - 2Y_i\beta_i + \beta_i^2}{2}\bigg)\propto\exp\bigg(-\dfrac{\beta_i^2}{2\tau_i^2} + Y_i\beta_i - \dfrac{\beta_i^2}{2}\bigg)$$ I don't think I am; the expectation of a constant times a normal distribution is not the same as the expectation of the normal distribution itself right? I thought I could mention it anyway since I might be mistaken.


You definitely went in the right direction. If we start form your second equation we find that $$ \dfrac{1}{2\pi}\exp\bigg(-\dfrac{\beta_i^2}{2\tau_i^2} - \dfrac{Y_i^2 - 2Y_i\beta_i + \beta_i^2}{2}\bigg) \propto \exp\bigg(-\frac{1}{2} \frac{\tau_i^2+1}{\tau_i^2}\left(\beta_i - \frac{\tau_i^2}{\tau_i^2+1}Y_i\right)^2 \bigg)$$ From there you can easily derive the distribution of $\beta \mid Y,\tau$, which is normal and thus $$\beta \mid Y,\tau \sim N\left(D(D+I)^{-1}Y, D(D+I)^{-1}\right) $$

  • $\begingroup$ Hi Tim thanks for your reply! Could you elaborate a bit more how you would continue and perhaps address the edit I made in my question? $\endgroup$ – Mr. President Apr 25 '18 at 15:34
  • $\begingroup$ Since $\beta_i\mid Y_i, \tau_i$ is normally distributed with known mean, you can use this to find $E[\beta_i\mid Y_i, \tau_i]$ (the expression I gave is more general). Regarding your edit, you are allowed to "throw away" factors that do not depend on $\beta$ since they do not determine the distribution. You can always find the "correct" distribution in the end by normalizing. $\endgroup$ – Tim Dikland Apr 25 '18 at 15:43
  • $\begingroup$ I really appreciate your answer but it's just too advanced for me now! How would I find the correct distribution by normalizing after I've thrown away the constants? Could you show me? $\endgroup$ – Mr. President Apr 25 '18 at 18:06
  • $\begingroup$ When ignoring a constant factor in the distibution, the distribution does not integrate to 1 anymore. This can be fixed by dividing through the integral of the distibution (normalization), which retrieves the ignored constant factor. $\endgroup$ – Tim Dikland Apr 26 '18 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.