Let $X_{ij} ~ N(\theta_i,\sigma^2)$ with $\sigma^2$ known, i = 1,... k, and j = 1, ... ,$n_i$. The prior distribution of $ \theta_i$ is $N(\phi,\tau^2)$, independently for i = 1,...,k and $(\phi,\tau^2)$ is has improper prior distribution given by $g(\phi,\tau^2)$= constant, with $ -\infty< \phi <\infty$ and $\tau>0$.

a) Let $\theta=(\theta_1,...,\theta_k)$. Find the joint posterior distribution of $\theta,\phi,\tau^2$ (up to a normalising constant

b) Deduce the posterior distribution of $\theta$.

I understand that for (b) we integrate the posterior $\pi(\theta,\phi,\tau^2|x)$ from (a) w.r.t. $\phi$ and $\tau^2$. However, the solution integrates w.r.t. $\tau$, not $\tau^2$. Why is this the case, and sceondly in general how do we know what function to integrate w.r.t. in such cases?

  • $\begingroup$ Why do you think "we must integrate the posterior" w.r.t. $\tau^2$? What is wrong with parameter $\tau$? Marginalization over some subset of parameters in your case produces a posterior distribution. You always marginalize over the parameters that you do not want to see in your posterior (in your case it is $\phi$ and $\tau$) $\endgroup$ – Tomas Mar 17 '15 at 12:39
  • $\begingroup$ Sorry I don't understand your comment. I have edited above post to remove "must" as that is where I am unsure. My question is why do you integrate w.r.t $\tau$ and not $\tau^2$? I don't see why we can't treat the variance $\beta:=\tau^2$ as the unknown parameter, and have to use $\tau$ instead to marginalise over. $\endgroup$ – a.e. Mar 17 '15 at 13:36
  • $\begingroup$ ^above comment @Tomas , or anyone else who can help. $\endgroup$ – a.e. Mar 17 '15 at 14:18

The reason why it matters whether we integrating over $\tau$ or $\tau^2$ lies in the theory of random variable transformation. In your case the original variable is $\tau$, even thought in the posterior it appears only in the form of $\tau^2$. If you want to treat $\tau^2$ as your variable you must adjust your posterior for this transformation.

Your posterior is $\pi\left (\theta, \tau, \phi \right )$. The solution given to you integrates over the parameter $\tau$ from $0$ to $+\infty$. Since in your posterior parameter $\tau$ only appears in the form of $\tau^2$ it is quite natural to consider $\tau^2$ as your new parameter. Therefore we perform a transformation of random variable, i.e. your new parameter is $\varphi\left ( \tau \right )=\tau^2$. Because you introduced a transformation your posterior distribution needs an adjustment:

$$\pi_n\left (\theta, \varphi, \phi \right )=\pi\left (\theta, \sqrt\varphi, \phi \right )\cdot\left | \frac{\mathrm{d} }{\mathrm{d} \varphi} \sqrt{\varphi}\right |$$

In other words just put $\varphi$ instead of $\tau^2$ in your posterior and multiply it by $\frac{1}{2\sqrt{\varphi}}$. Now you can integrate over your new variable $\varphi$.

I would strongly suggest reading some probability textbook containing general information about the random variable transformations to get a clear view.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.