0
$\begingroup$

When solving for Ordinary Least Squares regression line I was taught that you solve the system of equations $$ y = X\beta \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(1)$$ where $X$ is the design matrix. The way that was shown to solve for it is $$ X^Ty = X^TX\beta$$ $$ (X^TX)^{-1}X^Ty = \beta $$

But now that I am looking at it more closely, doesn't equation $(1)$ not have a solution in almost all cases, since there is most certainly not a line that passes through all the points?

So I am wondering how solving using the transpose and inverse leads to a solution for the minimization of $\epsilon$ when it seems there is no solution to the system.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.