One of my lectures includes the following quote from my professor (on a part of a chapter about compatible systems):

$A^TA$ is semi-positive-definite. If columns of $A$ are linear independent, then $A^TA$ is positive definite and invertible and has only one solution.

I'm fine with the first part of it that it's positive semi-definite since

$\forall x(\neq 0): X^T(A^TA)X=\big\|AX\big\|_2^2 \geq0$

I don't get how from its columns being linearly independent we get to it's being PD.

  • 1
    $\begingroup$ You mean that $||AX||_2^2 \geq 0$, not $>0$. $\endgroup$ – bourbaki4481472 Dec 19 '13 at 23:28
  • $\begingroup$ @ijkilchenko: No, I mean what I've written above. It's from the definition of positive semi-definiteness. $\endgroup$ – Gigili Dec 19 '13 at 23:46
  • $\begingroup$ I presume you mean "$A^T A$ is positive-definite and invertible?" $\endgroup$ – Branimir Ćaćić Dec 19 '13 at 23:47
  • $\begingroup$ @BranimirĆaćić: You're right, thank you. $\endgroup$ – Gigili Dec 19 '13 at 23:48
  • 2
    $\begingroup$ There is a geometrical interpretation of this. The determinant of $A^TA$ equals the squared volume of the parallelogram spanned by the columns of $A$. This is nonzero exactly when those columns are linearly independent, for rather intuitive geometrical reasons. In this case the system $A^TAx=y$ has one and only one solution by Cramer's rule. $\endgroup$ – Giuseppe Negro Dec 19 '13 at 23:54

Exactly from the line you have written.
Observe that if the columns of $A$ are ${\bf a}_1,\dots,{\bf a}_n$ then with ${\bf x}=\pmatrix{x_1\\x_2\\ \vdots \\ x_n }$ we have $$A{\bf x}=x_1{\bf a}_1+x_2{\bf a}_2+\dots+x_n{\bf a}_n\,.$$ So, if ${\bf a}_i$'s are linearly independent, that means that $A{\bf x}=0\implies {\bf x}=0$.

  • $\begingroup$ Could you check my answer, please? $\endgroup$ – bourbaki4481472 Dec 19 '13 at 23:44

I think I have a counterexample to your professor's claim. Let A= $ \left( \begin{matrix} 0 & 1 \\ -1 & 0 \\ \end{matrix} \right)$; clearly, $A$ is linearly independent. And let $x^T=(1 \;\; 2)$. $$x^T A x=$$ $$\left(1 \;\; 2\right)\left( \begin{matrix} 0 & 1 \\ -1 & 0 \\ \end{matrix} \right)\left( \begin{matrix} 1\\ 2\\ \end{matrix} \right)=$$ $$\left(1 \;\; 2\right)\left( \begin{matrix} 2\\ -1\\ \end{matrix} \right)=0.$$ Where am I wrong here?

  • $\begingroup$ OP changed the question. This doesn't answer the question anymore. $\endgroup$ – bourbaki4481472 Dec 20 '13 at 0:01
  • 2
    $\begingroup$ It is about $x^T{\bf A}^TAx$ not $x^TAx$. $\endgroup$ – Berci Dec 20 '13 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.