When solving for Ordinary Least Squares regression line I was taught that you solve the system of equations $$ y = X\beta \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(1)$$ where $X$ is the design matrix. The way that was shown to solve for it is $$ X^Ty = X^TX\beta$$ $$ (X^TX)^{-1}X^Ty = \beta $$

But now that I am looking at it more closely, doesn't equation $(1)$ not have a solution in almost all cases, since there is most certainly not a line that passes through all the points?

So I am wondering how solving using the transpose and inverse leads to a solution for the minimization of $\epsilon$ when it seems there is no solution to the system.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.