Obtain the conditional distributions from the full posterior distribution

From the Posterior distribution below I would like to obtain the distributions $\mu|\sigma^2,x$ and $\sigma^2|\mu,x$

The likelihood is given by

$\prod L(x_i|\mu,\sigma^2) \propto \sigma^{-20}\prod_{i=1}^{20} exp(- \frac{(x_i-\mu)^2}{2\sigma^2 \cdot u_i})$ (A)

Where $X_i \sim N(\mu,\sigma^2 \cdot u_i^2)$

The prior is proportional to:

$f(\mu,\sigma^2)\propto \sigma^{-22}exp(\frac{(\mu-2)^2 + 20\cdot 2\cdot 1.7^2}{2\sigma^2 \cdot 1.7^2})$ (B)

And the posterior is therefore given by

$f(\mu,\sigma^2|x) \propto \sigma^{-22}exp(\frac{(\mu-2)^2 + 20\cdot 2\cdot 1.7^2}{2\sigma^2 \cdot 1.7^2})\cdot \sigma^{-20}\cdot exp(- \frac{\sum_{i=1}^{i=5} (x_i-\mu)^2}{\sigma^2 }) \cdot exp(- \frac{\sum_{i=6}^{i=10} (x_i-\mu)^2}{2\sigma^2 }) \cdot exp(- \frac{\sum_{i=11}^{i=15} (x_i-\mu)^2}{3\sigma^2 }) \cdot exp(- \frac{\sum_{i=16}^{i=20} (x_i-\mu)^2}{4\sigma^2 }.).$

But to get eg. $f(\mu|\sigma^2,x)= \frac{f(\mu,\sigma^2|x)}{f(\sigma^2|x)}$ I need to obtain $f(\sigma^2|x)$. I guess that I have to integrate out $\mu$ but how that is going to be done I am not sure

Case of $\mathbf{P}(\mu,\ | \,\sigma^2 \, x)$

First of all

\begin{align*} \mathbf{P}(\mu,\sigma^2 \, | \, x) & \propto \mathbf{P}(x \, | \, \mu,\sigma^2 ) \mathbf{P}(\mu, \sigma^2) \\ & \propto \mathbf{P}(x \, | \, \mu,\sigma^2 ) \mathbf{P}(\mu \, | \, \sigma^2)\mathbf{P}(\sigma^2) \end{align*}

Then, we are interested in computing

\begin{align*} \mathbf{P}(\mu\, | \,\sigma^2, \, x) & = \frac{ \mathbf{P}(\mu, \, \sigma^2 \, | \, x )}{\mathbf{P}(\sigma^2\, | \, x)} \\ & \propto \mathbf{P}(\mu, \, \sigma^2 \, | \, x ) \end{align*} which is true since the left hand side is a function in $\mu$ only ($\sigma^2$ is now fixed), and so $\mathbf{P}(\sigma^2\, | \, x)$ is a constant. This further simplifies to

$$\mathbf{P}(\mu\, | \,\sigma^2, \, x) \propto \mathbf{P}(x\, | \,\mu, \,\sigma^2)\mathbf{P}(\mu\, | \,\sigma^2).$$

Now, the terms on the right hand side are all known, we have: \begin{align*} \mathbf{P}(\mu \, | \, \sigma^2,\,x) & \propto \exp \left( -\frac12 \sum_{i=1}^n \frac{(x_i - \mu)^2}{\sigma^2 u_i^2} \right) \times \exp \left( -\frac12 \frac{(2 - \mu)^2}{\sigma^2 1.7^2}\right)\\ & \propto \exp \left( -\frac12 \sum_{i=0}^n \frac{(x_i - \mu)^2}{\sigma^2 u_i^2} \right) \end{align*}

where $n = 20$, and in the final step we add the additional term $i = 0$ with $x_i = 2$ and $u_i = 1.7$.

We can now recognise this as the form for a product of independent Normal distributions, with

$$\mathbf{P}(\mu \, | \, \sigma^2,\,x) = \bigotimes_{i=0}^n \, {\large N}\left(x_i, \sigma^2 u_i^2\right),$$ where I used $\otimes$ just to denote that it is a product of independent univariate distributions.

Case of $\mathbf{P}(\sigma^2 \, | \, \mu, \, x)$

As in the above, we use the same identity for $\mathbf{P}(\mu,\, \sigma^2 \, | \, x)$ to derive

\begin{align*} \mathbf{P}(\sigma^2 \, | \, \mu,\, x) & \propto P(x \, | \, \mu, \, \sigma^2) \mathbf{P}(\mu \, | \, \sigma^2) \mathbf{P}(\sigma^2) \\ & \propto \frac{1}{\sigma^{n}} \exp \left(- \frac12 \sum_{i=1}^n \frac{(x_i - \mu)^2}{\sigma^2 u_i^2} \right) \times \frac{1}{\sigma} \exp \left( -\frac12 \frac{(2 - \mu)^2}{\sigma^2 1.7^2}\right) \times \frac{1}{\sigma^{22}} \exp\left( -\frac{20}{\sigma^2} \right) \\ & \propto \frac{1}{(\sigma^2)^{(23 + n)/2}} \exp\left( -\frac{1}{\sigma^2} \left(20 + \sum_{i=0}^n \frac{(x_i - \mu)^2}{u_i^2} \right) \right) \\ & \propto \frac{1}{(\sigma^2)^{A}} \exp\left(- \frac{B}{\sigma^2} \right) \end{align*} where $$A = \frac{23 + n}{2}, \qquad B = 20 + \sum_{i=0}^n \frac{(x_i - \mu)^2}{u_i^2}$$ are constants (as a function of $\sigma^2$ only). Note that again I assumed at $i=0$, $x_i = 2$, $u_i = 1.7$.

Therefore we recognise the posterior distribution as an Inverse Gamma

$$\mathbf{P}(\sigma^2 \, | \, \mu, \, x) = \text{InvGamma}(A, B).$$

• From $P(\mu|\sigma^2,x) \propto \prod N(x_i,\sigma^2u_i^2)$ is it possible to find the distribution $\mu|\sigma^2,x$ Because I want to make a random draw from this distribution but how do I do that? @owen88 – Danny Jan 20 '18 at 12:30