0
$\begingroup$

I'm doing Bayesian problem where I need to calculate a posterior function for distribution mean $\theta$.

The problem goes as follows: I have a data set $y_1, y_2, ..., y_n$ with each $y_i \sim N(\theta, \sigma^2 = 40).$ I'm given that $\frac1n\sum_{i=1}^n y_i = 150$ and that the prior distribution for $\theta$ is $p(\theta)=N(\mu_0=180, \tau_0^2=160).$ Derive the posterior distribution for $\theta$, i.e. $p(\theta|y)$.

I have no problem in this task except with evaluating the normalizing constant:

\begin{align}p(y)&=\int_{-\infty}^{\infty}p(y|\theta)p(\theta)\,d\theta\\&=\frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}\left(\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2\right)\right)\,d\theta \end{align}

My question is: how can I evaluate the Gaussian integral above?

$\endgroup$
1
$\begingroup$

\begin{align}p(y)&=\int_{-\infty}^{\infty}p(y|\theta)p(\theta)\,d\theta\\&=\frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}\left(\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2\right)\right)\,d\theta \end{align}

Notice that

$$\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2 $$

is a quadratic function with leading coefficient being positive in $\theta$ and hence we can write it in the form of $$\frac{(\theta -B)^2}{A^2} + C$$

Hence \begin{align}p(y)&= \frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{(\theta -B)^2}{2A^2} -\frac{C}{2}\right)\,d\theta \\&= \frac{\exp(-C/2)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{(\theta -B)^2}{2A^2} \right)\,d\theta \\ &=\frac{\exp(-C/2)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \sqrt{2\pi A^2}\end{align}

Hence if it suffices to solve for $A$ and $C$ to solve for your problem.

$\endgroup$
  • $\begingroup$ Thank you for your help! =) $\endgroup$ – jjepsuomi Sep 16 '17 at 14:57
  • $\begingroup$ Hi @SiongThyeGoh based on your answer I tried solving it myself also. Could you quickly check on it and verify whether I'm mistaken about my result? Thank you $\endgroup$ – jjepsuomi Sep 17 '17 at 10:56
  • $\begingroup$ second line $B$ and $C$ should have term involving $\tau_0$. $\endgroup$ – Siong Thye Goh Sep 17 '17 at 11:02
  • $\begingroup$ Thank you, damn I missed that x) $\endgroup$ – jjepsuomi Sep 17 '17 at 11:05
  • $\begingroup$ I think now I got it. $\endgroup$ – jjepsuomi Sep 17 '17 at 11:16
0
$\begingroup$

By using the answer already given I tried solving it myself also :)

$$\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2 =\frac{1}{\tau_0^2}(\theta^2-2\theta\mu_0+\mu_0^2)+\frac{1}{\sigma^2}\left(n\theta^2-2\theta \sum_{i=1}^n y_i+\sum_{i=1}^n y_i^2\right) $$

$$\underbrace{\left(\frac{1}{\tau_0^2}+\frac{n}{\sigma^2}\right)}_{=A}\theta^2-2\underbrace{\left(\frac{\mu_0}{\tau_0^2}+\frac{\sum_{i=1}^n y_i}{\sigma^2}\right)}_{=B}\theta+\underbrace{\left(\frac{\mu_0^2}{\tau_0^2}+\frac{1}{\sigma^2}\sum_{i=1}^n y_i^2\right)}_{=C}=A\theta^2-2B\theta+C,$$

so now I have:

$$\begin{align} p(y) &=\int_{-\infty}^{\infty}p(y|\theta)p(\theta)\,d\theta=\frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}\left(\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2\right)\right)\,d\theta \\ &= \frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac A2\theta^2+B\theta-\frac C2\right)\,d\theta \\ &= \frac{\exp\left(-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac A2\theta^2+B\theta\right)\,d\theta \\&= \frac{\exp\left(-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-\frac A2\left(\theta-\frac BA\right)^2 +\frac{B^2}{2A}\right)\,d\theta \\&= \frac{\exp\left(\frac{B^2}{2A}-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-\frac A2\left(\theta-\frac BA\right)^2 \right)\,d\theta\end{align}. $$

Now I set $F=\frac A2\left(\theta-\frac BA\right), dF=\frac A2 d\theta$ so I get:

$$\begin{align} &=\frac{\exp\left(\frac{B^2}{2A}-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-\frac A2\left(\theta-\frac BA\right)^2 \right)\,d\theta =\frac{2\exp\left(\frac{B^2}{2A}-\frac C2\right)}{A\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-F^2 \right)\,dF \\&= \frac{2\exp\left(\frac{B^2}{2A}-\frac C2\right)}{A\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\sqrt{\pi}\\&= \frac{2\sqrt{\pi}\exp\left(\frac{\left(\frac{\mu_0}{\tau_0^2}+\frac{\sum_{i=1}^n y_i}{\sigma^2}\right)^2}{2\left(\frac{1}{\tau_0^2}+\frac{n}{\sigma^2}\right)}-\frac{\left(\frac{\mu_0^2}{\tau_0^2}+\frac{1}{\sigma^2}\sum_{i=1}^n y_i^2\right)}{2}\right)}{\left(\frac{1}{\tau_0^2}+\frac{n}{\sigma^2}\right)\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}.\end{align}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.