Revisit: If $A$ is full column rank, then $A^TA$ is always invertible

Question

This is possibly a stupid question. I've read the question and answer in this link If $A$ is full column rank, then $A^TA$ is always invertible already.

My question is from $x=0$ how can we conclude $A^TA$ is invertible (or nonsingular)? I hope to get a simple (uncomplicated) explanation. Many thanks!

Answer 1

If the columns $c_1, \ldots, c_n$ of an $n \times n$ matrix $B$ do not generate the whole space, then they must be linearly dependent. Thus, there exist $x_1, \ldots,x_n$ not all zero for which

$$ x_1c_1 + \ldots + x_n c_n = 0. $$

In other words, if $x := (x_1, \ldots, x_n)$ then $x \neq 0$ and

$$ Bx = 0. $$

This way, we see that if $B$ does not have full rank, then there exits $x \neq 0$ for which $Bx = 0$.

The answer you cite shows that if $A^T Ax = 0$, then $x = 0$, hence $A^T A$ must have full rank by our previous remark.

Now, if a matrix $B$ has full rank, it is invertible: for each $e_i = (0,\ldots, \overbrace{1}^i,\ldots 0)$ there exists $(a_{ij})_i$ for which

$$ d_{1i}c_1 + \ldots + d_{ni}c_n = e_i, $$

since $c_1, \ldots, c_n$ generate the whole space. If $C$ is the matrix defined by $D_{ij} = d_{ij}$, we see that

$$ BD = I. $$

This in turn shows that $B$ is invertible.