2
$\begingroup$

From the Posterior distribution below I would like to obtain the distributions $\mu|\sigma^2,x$ and $\sigma^2|\mu,x$

The likelihood is given by

$\prod L(x_i|\mu,\sigma^2) \propto \sigma^{-20}\prod_{i=1}^{20} exp(- \frac{(x_i-\mu)^2}{2\sigma^2 \cdot u_i}) $ (A)

Where $X_i \sim N(\mu,\sigma^2 \cdot u_i^2)$

The prior is proportional to:

$f(\mu,\sigma^2)\propto \sigma^{-22}exp(\frac{(\mu-2)^2 + 20\cdot 2\cdot 1.7^2}{2\sigma^2 \cdot 1.7^2}) $ (B)


And the posterior is therefore given by

$f(\mu,\sigma^2|x) \propto \sigma^{-22}exp(\frac{(\mu-2)^2 + 20\cdot 2\cdot 1.7^2}{2\sigma^2 \cdot 1.7^2})\cdot \sigma^{-20}\cdot exp(- \frac{\sum_{i=1}^{i=5} (x_i-\mu)^2}{\sigma^2 }) \cdot exp(- \frac{\sum_{i=6}^{i=10} (x_i-\mu)^2}{2\sigma^2 }) \cdot exp(- \frac{\sum_{i=11}^{i=15} (x_i-\mu)^2}{3\sigma^2 }) \cdot exp(- \frac{\sum_{i=16}^{i=20} (x_i-\mu)^2}{4\sigma^2 }.). $

But to get eg. $f(\mu|\sigma^2,x)= \frac{f(\mu,\sigma^2|x)}{f(\sigma^2|x)}$ I need to obtain $f(\sigma^2|x)$. I guess that I have to integrate out $\mu$ but how that is going to be done I am not sure

$\endgroup$
2
$\begingroup$

Case of $\mathbf{P}(\mu,\ | \,\sigma^2 \, x)$

First of all

\begin{align*} \mathbf{P}(\mu,\sigma^2 \, | \, x) & \propto \mathbf{P}(x \, | \, \mu,\sigma^2 ) \mathbf{P}(\mu, \sigma^2) \\ & \propto \mathbf{P}(x \, | \, \mu,\sigma^2 ) \mathbf{P}(\mu \, | \, \sigma^2)\mathbf{P}(\sigma^2) \end{align*}

Then, we are interested in computing

\begin{align*} \mathbf{P}(\mu\, | \,\sigma^2, \, x) & = \frac{ \mathbf{P}(\mu, \, \sigma^2 \, | \, x )}{\mathbf{P}(\sigma^2\, | \, x)} \\ & \propto \mathbf{P}(\mu, \, \sigma^2 \, | \, x ) \end{align*} which is true since the left hand side is a function in $\mu$ only ($\sigma^2$ is now fixed), and so $\mathbf{P}(\sigma^2\, | \, x)$ is a constant. This further simplifies to

$$ \mathbf{P}(\mu\, | \,\sigma^2, \, x) \propto \mathbf{P}(x\, | \,\mu, \,\sigma^2)\mathbf{P}(\mu\, | \,\sigma^2). $$

Now, the terms on the right hand side are all known, we have: \begin{align*} \mathbf{P}(\mu \, | \, \sigma^2,\,x) & \propto \exp \left( -\frac12 \sum_{i=1}^n \frac{(x_i - \mu)^2}{\sigma^2 u_i^2} \right) \times \exp \left( -\frac12 \frac{(2 - \mu)^2}{\sigma^2 1.7^2}\right)\\ & \propto \exp \left( -\frac12 \sum_{i=0}^n \frac{(x_i - \mu)^2}{\sigma^2 u_i^2} \right) \end{align*}

where $n = 20$, and in the final step we add the additional term $i = 0$ with $x_i = 2$ and $u_i = 1.7$.

We can now recognise this as the form for a product of independent Normal distributions, with

$$ \mathbf{P}(\mu \, | \, \sigma^2,\,x) = \bigotimes_{i=0}^n \, {\large N}\left(x_i, \sigma^2 u_i^2\right), $$ where I used $\otimes$ just to denote that it is a product of independent univariate distributions.


Case of $\mathbf{P}(\sigma^2 \, | \, \mu, \, x)$

As in the above, we use the same identity for $\mathbf{P}(\mu,\, \sigma^2 \, | \, x)$ to derive

\begin{align*} \mathbf{P}(\sigma^2 \, | \, \mu,\, x) & \propto P(x \, | \, \mu, \, \sigma^2) \mathbf{P}(\mu \, | \, \sigma^2) \mathbf{P}(\sigma^2) \\ & \propto \frac{1}{\sigma^{n}} \exp \left(- \frac12 \sum_{i=1}^n \frac{(x_i - \mu)^2}{\sigma^2 u_i^2} \right) \times \frac{1}{\sigma} \exp \left( -\frac12 \frac{(2 - \mu)^2}{\sigma^2 1.7^2}\right) \times \frac{1}{\sigma^{22}} \exp\left( -\frac{20}{\sigma^2} \right) \\ & \propto \frac{1}{(\sigma^2)^{(23 + n)/2}} \exp\left( -\frac{1}{\sigma^2} \left(20 + \sum_{i=0}^n \frac{(x_i - \mu)^2}{u_i^2} \right) \right) \\ & \propto \frac{1}{(\sigma^2)^{A}} \exp\left(- \frac{B}{\sigma^2} \right) \end{align*} where $$A = \frac{23 + n}{2}, \qquad B = 20 + \sum_{i=0}^n \frac{(x_i - \mu)^2}{u_i^2}$$ are constants (as a function of $\sigma^2$ only). Note that again I assumed at $i=0$, $x_i = 2$, $u_i = 1.7$.

Therefore we recognise the posterior distribution as an Inverse Gamma

$$\mathbf{P}(\sigma^2 \, | \, \mu, \, x) = \text{InvGamma}(A, B).$$

$\endgroup$
  • $\begingroup$ From $P(\mu|\sigma^2,x) \propto \prod N(x_i,\sigma^2u_i^2)$ is it possible to find the distribution $\mu|\sigma^2,x$ Because I want to make a random draw from this distribution but how do I do that? @owen88 $\endgroup$ – Danny Jan 20 '18 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.