I am studying on some linear regression problems using the least-squares method and stumbled upon a problem regarding the error when extrapolating far-away datapoints.
About the problem: For a point $y$ given by best fitting line, described by
$y = ax + b$,
where $a$ and $b$ are the coefficients achieved by applying the least-squares-method to a given data set, we know that generally, the error or the variance for $y$, will increase, the further we move away from our actual data, while the variance will be minimal at the mean position of $x$ of our given data set. This is so far totally clear for me.
But now I wondered:
Say, I have a given data set that consists of two clusters, with many datapoints around a very small negative position $j$ and another cluster at a very big positive position $k$.
Wouldn't the overall error of the best fitting line be decreased now? Will our line, for example for interpolating points in between, be a better fit with this kind of measurement, than a line that comes from only one data cluster int the 'middle' of the two-cluster version?
I hope I explained my problem sufficiently and I am excited for your suggestions!