I'm reading about Bayesian Data Analysis (by Gelman et al. 3rd edition, page 167-168) and there is one part I can't understand:

Predictive accuracy for a single data point

The ideal measure of a model’s fit would be its out-of-sample predictive performance for new data produced from the true data-generating process (external validation). We label $f$ as the true model, $y$ as the observed data (thus, a single realization of the dataset $y$ from the distribution $f(y)$), and $\tilde{y}$ as future data or alternative datasets that could have been seen. The out-of-sample predictive fit for a new data point $\tilde{y}_i$ using logarithmic score is then,

$$ \log p_{post}(\tilde{y}_i) = \log E_{post}(p(\tilde{y}_i\,|\,\theta)) = \log \int p(\tilde{y}_i\,|\,\theta)\, p_{post}(\theta)\,d\theta. $$

In the above expression, $p_{post}(\tilde{y}_i)$ is the predictive density for $\tilde{y}_i$ induced by the posterior distribution $p_{post}(\theta)$. We have introduced the notation $p_{post}$ here to represent the posterior distribution because our expressions will soon become more complicated and it will be convenient to avoid explicitly showing the conditioning of our inferences on the observed data $y$. More generally, we use $p_{post}$ and $E_{post}$ to denote any probability or expectation that averages over the posterior distribution of $\theta$.

Averaging over the distribution of future data

We must then take one further step. The future data $\tilde{y}_i$ are themselves unknown and thus we define the expected out-of-sample log predictive density, $$ \text{elpd} = \text{expected log predictive density for a new data point}$$ $$ = E_f (\log p_{post}(\tilde{y}_i)) = \int \log p_{post}(\tilde{y}_i)\,f(\tilde{y}_i)\,d\tilde{y}.$$

What I don't understand is the last equation. What is $$E_f (\log p_{post}(\tilde{y}_i)) = \int \log p_{post}(\tilde{y}_i)\,f(\tilde{y}_i)\,d\tilde{y}?$$

What does $f(y)$ mean? This is confusing notation? Does $f(y)$ mean: "Distribution function which generated data $y$" or what?

Thank you for your help.


I'm trying to figure this out myself, could it be that the:

$$ \text{elpd} = E_f (\log p_{post}(\tilde{y}_i)) = \int \log p_{post}(\tilde{y}_i)\,f(\tilde{y}_i)\,d\tilde{y}$$

simply means that, given that on future we will observe some data $\tilde{y}$ with distribution $f$, then on average with respect to this future data the log predictive density for data point $\tilde{y}_i$ is $E_f (\log p_{post}(\tilde{y}_i))$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.