4
$\begingroup$

I am having some trouble understanding the basic definition of a posterior predictive distribution and how to apply it to simple examples.

I have that if $$y \sim g(y\mid \theta, x)$$

then the predictive posterior distribution is

$$g(y\mid x)=\int g(y\mid\theta,x)\pi(\theta\mid x) \,d\theta$$

So I am trying to work through an example of this where

$x \sim \operatorname{Normal}(\theta, \sigma^2)$ and $y \sim \operatorname{Normal}(ex,\sigma^2)$ and $\pi(\theta,\sigma^2)=\frac{1}{\sigma^2}$

And I am not sure I understand all the notations correctly. I don't have any examples either of actually solving for this, but I would be interested in seeing any.

So I know that $$\pi(\theta\mid x) \propto \pi(x\mid\theta) \pi(\theta)$$

so $$\pi(\theta,\sigma^2\mid x) \propto \exp\left(\frac{-1}{2\sigma^2}{(x-\theta)^2}\right) \frac{1}{\sigma^2}$$

But then I am not sure how to proceed.

I assume I am looking for $\pi(Y\mid X=x)$

Well $P(Y\mid X=x,\theta , \sigma^2) = \exp(\frac{-1}{2\sigma^2}(y-ex)^2)$

so,

$$P(Y\mid X=x,\theta , \sigma^2) \pi(\theta,\sigma^2 \mid x)= \frac{1}{\sigma^2} \exp\left(\frac{-1}{2\sigma^2}[(y-ex)^2+(x-\theta)^2]\right)$$

Note that if we try to double integral to normalise over $\pi(x\mid\theta , \sigma^{2})\pi(\theta,\sigma^2)$ we get a divergent integral. So we cannot make equality.

$\endgroup$
  • 1
    $\begingroup$ It is not clear what you are asking and what object you are trying to understand. Are you asking given the distribution $y \sim g(y| \theta, x)$ how one computes $g(y|x)$? If so, is it not just plugging into the defining integral and computing it? $\endgroup$ – Hans Oct 3 '18 at 8:57
  • $\begingroup$ @Hans But the integral diverges due to improper prior.. $\endgroup$ – Quality Oct 5 '18 at 20:34
  • $\begingroup$ "I am not sure I understand all the notations correctly." Some of the notation styles used in this question are unfortunately also used even in lots of published work, and they can be a barrier to understanding. (But I haven't finished digesting your question yet.) $\endgroup$ – Michael Hardy Oct 5 '18 at 21:59
  • $\begingroup$ You say $x\sim\operatorname{Normal}(\cdots),$ and then you use the same lower-case $x$ to refer to the argument to the density function, and at another point in your posting you write $X=x,$ as if clearly distinguishing between $X$ and $x.$ When one writes $f_X(x),$ distinguishing between those, then one can understand the difference between $f_X(3)$ and $f_Y(3)$ and the meaning of things like $\Pr(X\le x). \qquad$ $\endgroup$ – Michael Hardy Oct 5 '18 at 22:02
  • $\begingroup$ Thanks for the comment. I took the example directly as written from the notes. But I agree it is confusing $\endgroup$ – Quality Oct 5 '18 at 22:08
-1
+50
$\begingroup$

To sum up, you should know the prior distribution of $\sigma^2$. And, yes, you need double integral to integrate out both $\sigma^2, \theta$, unless $\sigma^2$ has been assumed to be a given constant.

Let's clarify what you are looking for. Your are looking for the posterior predictive, i.e. the distribution of $Y | X=x$. Your model is as follows. Given constant $e$,

$$ \begin{aligned} X \mid \theta, \sigma^2 &\sim N(\theta,\sigma^2) \\[7pt] Y \mid x, \theta, \sigma^2 &\sim N(ex,\sigma^2) \\[7pt] π(θ \mid \sigma^2) &= 1/\sigma^2 I_\Theta \\[7pt] \sigma^2 &\sim f_{\sigma^2} \end{aligned} $$

So you have

$$ \begin{aligned} \pi(\theta, \sigma^2 \mid x ) &\propto \pi(x \mid \theta, \sigma^2) \pi(\theta \mid \sigma^2) \pi(\sigma^2)\\[7pt] &\propto \exp(-\frac{1}{2\sigma^2} (x-\theta)^2) 1/\sigma^2 \cdot f_{\sigma^2} \end{aligned} $$

Thus

$$ \begin{aligned} P(Y \mid X=x) &= \int_{\sigma^2} \int_\theta P( Y, \theta, \sigma^2 \mid X=x) \\[7pt] &= \int_{\sigma^2} \int_\theta \pi(Y \mid x, \theta, \sigma^2) \pi (\theta \mid \sigma^2) f_{\sigma^2} \\[7pt] &= \int_{\sigma^2} \int_\theta N(ex,\sigma^2) 1/\sigma^2 f_{\sigma^2} \\[7pt] \end{aligned} $$

$\endgroup$
  • $\begingroup$ when you try to compute the integral of $ \pi(\theta,\sigma^{2} | x) $ the integral diverges. $\endgroup$ – Quality Oct 4 '18 at 22:16
  • $\begingroup$ So is $\pi(\theta, \sigma^2)$ the improper prior? Otherwise, it must have limited support range. $\endgroup$ – Moreblue Oct 5 '18 at 0:49
  • $\begingroup$ Well yea the prior is improper and that also ends up being what makes our double integral divergent $\endgroup$ – Quality Oct 5 '18 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.