I'm an undergrad that just got introduced to ordinary least squares and am trying to understand why a Gram matrix is invertible only if the column vectors are linearly independent. The thread below (3rd post ) uses the exact same proof as my lecturer. However, there is still something that is not clear for me which probably is something silly.
JasonMond's "only if" part is not as general as it should, because s/he assumes that AA is square. In the following, I complete the proof that holds whether AA is square or not.
Let $G=A^TA$. If the column vectors of $A$ are linearly dependent, there exists a vector $u≠0$ such that
$Au = 0$
It follows that
So i understand that if the column vectors of A are linearly dependent, then I can find a non-zero vector u such that $Au=0$. But how does this tell me that G is not invertible ? Because there are other vectors such that $Au\neq0$
For any square matrix G to be invertible, does it require the result of $Gu$ to be non-zero for all non-zero vectors $u$ ? If so, where can I read about this ?