# Predictive distribution with divergent integral

I am having some trouble understanding the basic definition of a posterior predictive distribution and how to apply it to simple examples.

I have that if $$y \sim g(y\mid \theta, x)$$

then the predictive posterior distribution is

$$g(y\mid x)=\int g(y\mid\theta,x)\pi(\theta\mid x) \,d\theta$$

So I am trying to work through an example of this where

$$x \sim \operatorname{Normal}(\theta, \sigma^2)$$ and $$y \sim \operatorname{Normal}(ex,\sigma^2)$$ and $$\pi(\theta,\sigma^2)=\frac{1}{\sigma^2}$$

And I am not sure I understand all the notations correctly. I don't have any examples either of actually solving for this, but I would be interested in seeing any.

So I know that $$\pi(\theta\mid x) \propto \pi(x\mid\theta) \pi(\theta)$$

so $$\pi(\theta,\sigma^2\mid x) \propto \exp\left(\frac{-1}{2\sigma^2}{(x-\theta)^2}\right) \frac{1}{\sigma^2}$$

But then I am not sure how to proceed.

I assume I am looking for $$\pi(Y\mid X=x)$$

Well $$P(Y\mid X=x,\theta , \sigma^2) = \exp(\frac{-1}{2\sigma^2}(y-ex)^2)$$

so,

$$P(Y\mid X=x,\theta , \sigma^2) \pi(\theta,\sigma^2 \mid x)= \frac{1}{\sigma^2} \exp\left(\frac{-1}{2\sigma^2}[(y-ex)^2+(x-\theta)^2]\right)$$

Note that if we try to double integral to normalise over $$\pi(x\mid\theta , \sigma^{2})\pi(\theta,\sigma^2)$$ we get a divergent integral. So we cannot make equality.

• It is not clear what you are asking and what object you are trying to understand. Are you asking given the distribution $y \sim g(y| \theta, x)$ how one computes $g(y|x)$? If so, is it not just plugging into the defining integral and computing it? – Hans Oct 3 '18 at 8:57
• @Hans But the integral diverges due to improper prior.. – Quality Oct 5 '18 at 20:34
• "I am not sure I understand all the notations correctly." Some of the notation styles used in this question are unfortunately also used even in lots of published work, and they can be a barrier to understanding. (But I haven't finished digesting your question yet.) – Michael Hardy Oct 5 '18 at 21:59
• You say $x\sim\operatorname{Normal}(\cdots),$ and then you use the same lower-case $x$ to refer to the argument to the density function, and at another point in your posting you write $X=x,$ as if clearly distinguishing between $X$ and $x.$ When one writes $f_X(x),$ distinguishing between those, then one can understand the difference between $f_X(3)$ and $f_Y(3)$ and the meaning of things like $\Pr(X\le x). \qquad$ – Michael Hardy Oct 5 '18 at 22:02
• Thanks for the comment. I took the example directly as written from the notes. But I agree it is confusing – Quality Oct 5 '18 at 22:08

To sum up, you should know the prior distribution of $$\sigma^2$$. And, yes, you need double integral to integrate out both $$\sigma^2, \theta$$, unless $$\sigma^2$$ has been assumed to be a given constant.

Let's clarify what you are looking for. Your are looking for the posterior predictive, i.e. the distribution of $$Y | X=x$$. Your model is as follows. Given constant $$e$$,

\begin{aligned} X \mid \theta, \sigma^2 &\sim N(\theta,\sigma^2) \\[7pt] Y \mid x, \theta, \sigma^2 &\sim N(ex,\sigma^2) \\[7pt] π(θ \mid \sigma^2) &= 1/\sigma^2 I_\Theta \\[7pt] \sigma^2 &\sim f_{\sigma^2} \end{aligned}

So you have

\begin{aligned} \pi(\theta, \sigma^2 \mid x ) &\propto \pi(x \mid \theta, \sigma^2) \pi(\theta \mid \sigma^2) \pi(\sigma^2)\\[7pt] &\propto \exp(-\frac{1}{2\sigma^2} (x-\theta)^2) 1/\sigma^2 \cdot f_{\sigma^2} \end{aligned}

Thus

\begin{aligned} P(Y \mid X=x) &= \int_{\sigma^2} \int_\theta P( Y, \theta, \sigma^2 \mid X=x) \\[7pt] &= \int_{\sigma^2} \int_\theta \pi(Y \mid x, \theta, \sigma^2) \pi (\theta \mid \sigma^2) f_{\sigma^2} \\[7pt] &= \int_{\sigma^2} \int_\theta N(ex,\sigma^2) 1/\sigma^2 f_{\sigma^2} \\[7pt] \end{aligned}

• when you try to compute the integral of $\pi(\theta,\sigma^{2} | x)$ the integral diverges. – Quality Oct 4 '18 at 22:16
• So is $\pi(\theta, \sigma^2)$ the improper prior? Otherwise, it must have limited support range. – Moreblue Oct 5 '18 at 0:49
• Well yea the prior is improper and that also ends up being what makes our double integral divergent – Quality Oct 5 '18 at 0:50