Why does the least squares regression line of $x$ on $y$ and $y$ on $x$ intersect at $\bar{x}$ and $\bar{y}$? Also, why are the form of regression lines as they are? For the general form $y-\bar{y}=b(x-\bar{x})$, what is $b$, and how is this derived?


If we have that $y$ is a dependant variable on $x$, then the least squares regression line is given as

$$ y - \bar{y} = \frac{\operatorname{Cov}[x,y]}{\operatorname{Var}[x]} (x-\bar{x}) $$

We can derive this formula by considering the optimization problem of minimizing the square of the residuals; more formally if we have a set of points $(x_1,y_1),(x_2,y_2), \ldots , (x_n,y_n)$ then the least squares regression line minimizes the function

$$ D(a,b) = \sum^n_{i=1} \epsilon_i^2 = \sum^n_{i=1} \left(y_i - [a + bx_i]\right)$$

so by solving for $\partial D / \partial a = 0$ and $\partial D / \partial b = 0$ simulataneously we get the above form.

Given the above form of the least squares regression, it should then become apparent why if we then take a least squares regression where $x$ is dependant on $y$ why the two lines intersect at $(\bar{x},\bar{y})$ - if it isn't immediate obvious, then recall the definition of $\operatorname{Cov}[x,y]$ and wlog. consider the case where $\bar{x}=\bar{y}=0$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.