# From positive semi-definiteness to positive definiteness

One of my lectures includes the following quote from my professor (on a part of a chapter about compatible systems):

$A^TA$ is semi-positive-definite. If columns of $A$ are linear independent, then $A^TA$ is positive definite and invertible and has only one solution.

I'm fine with the first part of it that it's positive semi-definite since

$\forall x(\neq 0): X^T(A^TA)X=\big\|AX\big\|_2^2 \geq0$

I don't get how from its columns being linearly independent we get to it's being PD.

• You mean that $||AX||_2^2 \geq 0$, not $>0$. – bourbaki4481472 Dec 19 '13 at 23:28
• @ijkilchenko: No, I mean what I've written above. It's from the definition of positive semi-definiteness. – Gigili Dec 19 '13 at 23:46
• I presume you mean "$A^T A$ is positive-definite and invertible?" – Branimir Ćaćić Dec 19 '13 at 23:47
• @BranimirĆaćić: You're right, thank you. – Gigili Dec 19 '13 at 23:48
• There is a geometrical interpretation of this. The determinant of $A^TA$ equals the squared volume of the parallelogram spanned by the columns of $A$. This is nonzero exactly when those columns are linearly independent, for rather intuitive geometrical reasons. In this case the system $A^TAx=y$ has one and only one solution by Cramer's rule. – Giuseppe Negro Dec 19 '13 at 23:54

Observe that if the columns of $A$ are ${\bf a}_1,\dots,{\bf a}_n$ then with ${\bf x}=\pmatrix{x_1\\x_2\\ \vdots \\ x_n }$ we have $$A{\bf x}=x_1{\bf a}_1+x_2{\bf a}_2+\dots+x_n{\bf a}_n\,.$$ So, if ${\bf a}_i$'s are linearly independent, that means that $A{\bf x}=0\implies {\bf x}=0$.
I think I have a counterexample to your professor's claim. Let A= $\left( \begin{matrix} 0 & 1 \\ -1 & 0 \\ \end{matrix} \right)$; clearly, $A$ is linearly independent. And let $x^T=(1 \;\; 2)$. $$x^T A x=$$ $$\left(1 \;\; 2\right)\left( \begin{matrix} 0 & 1 \\ -1 & 0 \\ \end{matrix} \right)\left( \begin{matrix} 1\\ 2\\ \end{matrix} \right)=$$ $$\left(1 \;\; 2\right)\left( \begin{matrix} 2\\ -1\\ \end{matrix} \right)=0.$$ Where am I wrong here?
• It is about $x^T{\bf A}^TAx$ not $x^TAx$. – Berci Dec 20 '13 at 0:19