# Linear Regression's Expectation of Prediction Error on a given point in the test set

I'm self-learning the book "The Elements of Statistical Learning", and I've got a little problem on deriving equation 2.27 in this book. I would appreciate if anyone could help me with this. In order to make this question consistent, I'll list all the information needed for the deriving.

Suppose we have two random variables $$Y$$ and $$X$$ with the following relation:

$$Y = X^T\beta + \epsilon$$ where $$\epsilon \sim N(0, \sigma^2)$$.

and we fit data in a training set $$\mathcal{T}$$ with linear regression by least squares:

$$\hat{Y} = X^T \hat{\beta}$$, where $$\hat{\beta}$$ is the parameter and $$\hat{Y}$$ is our prediction, $$X$$ is the input.

we define the Expected Prediction Error (EPE) of a record ($$x_0, y_0$$) in test data as

$$EPE(x_0) = E_{y_0|x_0}E_{\mathcal{T}}(y_0 - \hat{y_0})^2$$ where $$\hat{y_0}$$ is our prediction w.r.t. $$x_0$$.

According to the book , $$EPE(x_0) = Var(y_0|x_0) + Var_{\mathcal{T}}(\hat{y_0}) + Bias^2(\hat{y_0})$$

where $$Bias^2(\hat{y_0}) = (E_{\mathcal{T}}\hat{y_0} - y_0)^2$$.

Here is my derivation:

$$EPE(x_0) = E_{y_0|x_0}E_{\mathcal{T}}(y_0 - \hat{y_0})^2$$

$$=E_{y_0|x_0}E_{\mathcal{T}}(y_0^2 - 2y_0\hat{y_0} + \hat{y_0}^2)$$

$$=E_{y_0|x_0}(y_0^2) -2E_{y_0|x_0}y_0E_{\mathcal{T}}\hat{y_0} + E_{\mathcal{T}}\hat{y_0}^2$$

By the lemma

$$Var(X) = E(X^2) - (E(X))^2$$ we have

$$EPE(x_0) = Var(y_0|x_0) + (E_{y_0|x_0}(y_0))^2 -2E_{y_0|x_0}y_0E_{\mathcal{T}}\hat{y_0} + Var_{\mathcal{T}}(\hat{y_0)} + (E_{\mathcal{T}}(\hat{y_0}))^2$$

$$= Var(y_0|x_0) + Var_{\mathcal{T}}(\hat{y_0}) + (E_{\mathcal{T}}(\hat{y_0}) - E_{y_0|x_0}(y_0))^2$$ which is really close to the conclusion given in the book, as long as we can prove $$E_{y_0|x_0}(y_0) = y_0$$ the proof is completed.

But I've no clue how to prove $$E_{y_0|x_0}(y_0) = y_0$$ since I've no idea about $$p(y_0 | x_0)$$. I'm wondering if anyone could help me on that.

My guess is that, since $$y_0 = x_0^T\beta + \epsilon$$, $$E_{y_0|x_0}(y_0) = E_{y_0|x_0}(x_0^T\beta) + E_{y_0|x_0}(\epsilon) = E_{y_0|x_0}(x_0^T\beta) + \epsilon$$, and $$x_0^T\beta$$ is deterministic given $$x_0$$, so $$E_{y_0|x_0}(y_0) = x_0^T\beta + \epsilon = y_0$$, but I'm not sure if this is right.

Thank you very much for your reading!

Possible relative questions: