Why are $y^*$ and $\hat{y}^*$ independent? A common assertion that is made in several contexts, and used to prove some pretty important results, is that, because a given observation wasn't explicitly used in the creation of a certain statistical model, that the prediction given by that model is independent of that observation.
For example, assume $y$ can be modeled (the true model) on $x$ linearly by
\begin{equation*} 
y = f(x) + \varepsilon
\end{equation*}
for some linear function $f$ and random noise $\varepsilon$.  Let $\hat{f}$ be the least-squares estimate of $f$ one gets by linear regression on some collection of simultaneous observations of $y$ and $x$.  Let $(x^*,y^*)$ be an observation not part of the training data used to construct $\hat{f}$, and let $\hat{y}^* = \hat{f}(x^*)$, the value of $y$ predicted by your least-squares model.  In proving that, for example, that the distribution of $y^* - \hat{y}^*$ has mean $0$ and variance = (formula not important here), one encounters the assertion
"since $y^*$ was not used in the computation of $\hat{y}^*$, it follows that $y^*$ and $\hat{y}^*$ are independent, and hence $\text{Cov}(y^*,\hat{y}^*) = 0$."
You can find analogous assertions made in other contexts, for instance in the bias-variance trade-off theorem for statistical learning methods.
This assertion makes no sense to me.  Yes, I understand that $\hat{f}$ is not directly dependent on the value $y^*$ since it wasn't included in the training data.  But saying that they are independent is to say that knowing one should give you no better idea of the other.  This is obviously nonsense; what is the point of constructing and $\hat{f}$ to begin with if its value at $x^*$ gives you no better guess as to the value of $y^*$?
Is there an implicit conditional independence that's being implied here?  Because, as stated, this seems patently false.
Thank you in advance for any clarification.
 A: If x were random you'd need to talk about conditional independence.  But x is traditionally regarded as constant unless explicitly stated otherwise. So the independence assertion is correct.
And, yes, it IS confusing.
A: I agree the language is sloppy at best, but perhaps the following is what they mean?
Let $T=\{(x_i, y_i) : i \in \{1, 2, ..., N\}\}$ be the training data and $(x^*, y^*)$ be the new data.  Lets agree that the $N+1$ data points are independent.  I.e. there is some joint-probability $p(x,y)$ for each data point - and obviously you don't know what $p(x,y)$ is - but each data point is drawn according to it independently.  (One possible such joint probability is e.g. $y = f(x) + \varepsilon$ for some linear $f$ and noise model $\varepsilon$, e.g. gaussian $N(0,1)$, but this is not really necessary.)
Now, $\hat{f}$ is a deterministic function of $T$, and $\hat{y}^* = \hat{f}(x^*) $ is a deterministic function of $T$ and $x^*$.  This also means, conditioned on $x^*$ alone, $\hat{y}^*$ has a certain marginal distribution (with the "variability" coming from the random $T$).
Meanwhile, $y^*$ has a certain marginal distribution conditioned on the value $x^*$, based on $p(x,y)$.  Perhaps what they mean is: Conditioned on $x^*$, then $\hat{y}^*$ and $y^*$ are independent.  To be more precise, this means that, conditioned on $x^*=a$ (some specific value), if $T$ is such that you calculate $\hat{y}^*=b$ (some specific value), this does not affect the distribution of $y^*$, i.e. $y^*$ still has the same marginal distribution it has always had, which is $p(y|x=a)$.  The fact that you used to be ignorant, and is now more knowledgeable, is completely irrelevant - your knowledge/ignorance did not affect $y^*$ one bit.
In fact, for the specific case of $y = f(x) + \varepsilon$, and if we naturally assume that each sampling of $\varepsilon$ is independent, then conditioned on $x^* = a$ (some specific value), $y = f(a) + \varepsilon$ which is certainly independent of everything else.
What you had in mind is perhaps something like: "Before I built the model I had no idea of what $y^*$ would be, but now I have a good idea."  Specifically, suppose you build the model and calculate $\hat{y}^*=b$.  So now you say, before the model the "probability" that $y^* \approx b$ (lets say, defined as $|y^* - b| < 10$) is "low", but after the model the "probability" that $y^* \approx b$ is "high".  But in fact (according to the classical view), both before and after you build the model, the true probability that $y^* \approx b$ remains the same, and suppose it is actually high (i.e your model was good), then you were simply wrong before you build your model.  Another way to look at this is that before the model, you cannot even meaningfully guess that the probability is "low".
The scenario where your idea makes sense, is if there is some a-priori distribution on the space of all possible $p(x,y)$ itself.  A simpler, different example may illustrate this.  Suppose you have a coin with an unknown bias $p = Prob(H)$.  You wish to "learn" the bias and predict future flips.  So you flip the coin $N-1$ times, and calculate $M =$ the majority result among those flips, and use that to "predict" the next flip.  Are $M$ and the $N$th flip independent?  


*

*In the classical view, YES they are independent.  I.e., for any $p$ (the true model), $M$ and the $N$th flip are independent, and $P(Nth\ flip\ =H) = p$ regardless of whether there were more Heads before ($M=H$) or more Tails before ($M=T$).

*If however you are also given that $p$ is uniformly distributed within $(0,1)$, then NO, $M$ and the $N$th flip are no longer independent.  Once you know $p \sim U(0,1)$, you can meaningfully say, a-priori, that $P(Nth\ flip\ =H) = 1/2$ (i.e. averaged over all $p$), but you can also show (after some algebra) that $P(Nth\ flip\ = H | M=H) > 1/2$, so clearly $M$ and the $N$th flip are dependent.
You might argue, but we don't know $p$! So the classic view is "cheating"!  In reply, your opponent might say, but if you don't know $p$ (and are unwilling to assume something like $p \sim U(0,1)$), then you cannot even meaningfully compute $P(Nth\ flip\ =H)$ and $P(Nth\ flip\ =H | M=H)$, so how can you say they are unequal?
Hope this makes sense?
A: Thank you both, that cleared things up perfectly.  I think I got stuck in not realizing that $\hat{f}$, as a random variable, is a function of the training data, not of $x^*$, whereas $y^*$ is a function of $x^*$ (plus some additional randomness also independent from the training data).  Independence of $x^*$ from the training data makes this rather obvious.  Thanks again.
