To preface, this is a fairly basic linear algebra question, but I've been unsuccessful in finding a similar question on this site.
In the method of least squares for linear regression that is discussed in linear algebra, where the line is in the form of $y=Cx+D$ with $m$ data points, we set up an $m$ x 2 matrix ($A$) with the first column being the recorded x values and the second column being 1s multiplied by the vector with $C$ and $D$ as components ($X$) to get a column vector of the recorded y values ($b$). So $AX=b$.
In general, to solve these systems, since it is unlikely that all of the y values are in the column space of $A$, we typically solve the system $A^TAX=A^Tb$ instead, which is guaranteed to have a solution. This is where I am a bit confused. In the $A^TAX=A^Tb$ system, $AX$ is the projection, because the vector $AX$ minimizes the orthogonal distance between $AX$ and $b$. However, in linear regression, the squared vertical distance is minimized, not the orthogonal distance. Yet, linear regression still uses the $A^TAX=A^Tb$ method of solving the equations and clearly relies on the notion of projections. What accounts for the difference where projections minimize orthogonal distance and linear regression minimizes (squared) vertical distances?
Thanks.