1
$\begingroup$

One of my lectures includes the following quote from my professor (on a part of a chapter about compatible systems):

$A^TA$ is semi-positive-definite. If columns of $A$ are linear independent, then $A^TA$ is positive definite and invertible and has only one solution.

I'm fine with the first part of it that it's positive semi-definite since

$\forall x(\neq 0): X^T(A^TA)X=\big\|AX\big\|_2^2 \geq0$

I don't get how from its columns being linearly independent we get to it's being PD.

$\endgroup$
8
  • 1
    $\begingroup$ You mean that $||AX||_2^2 \geq 0$, not $>0$. $\endgroup$ Commented Dec 19, 2013 at 23:28
  • $\begingroup$ @ijkilchenko: No, I mean what I've written above. It's from the definition of positive semi-definiteness. $\endgroup$
    – Gigili
    Commented Dec 19, 2013 at 23:46
  • $\begingroup$ I presume you mean "$A^T A$ is positive-definite and invertible?" $\endgroup$ Commented Dec 19, 2013 at 23:47
  • $\begingroup$ @BranimirĆaćić: You're right, thank you. $\endgroup$
    – Gigili
    Commented Dec 19, 2013 at 23:48
  • 2
    $\begingroup$ There is a geometrical interpretation of this. The determinant of $A^TA$ equals the squared volume of the parallelogram spanned by the columns of $A$. This is nonzero exactly when those columns are linearly independent, for rather intuitive geometrical reasons. In this case the system $A^TAx=y$ has one and only one solution by Cramer's rule. $\endgroup$ Commented Dec 19, 2013 at 23:54

2 Answers 2

2
$\begingroup$

Exactly from the line you have written.
Observe that if the columns of $A$ are ${\bf a}_1,\dots,{\bf a}_n$ then with ${\bf x}=\pmatrix{x_1\\x_2\\ \vdots \\ x_n }$ we have $$A{\bf x}=x_1{\bf a}_1+x_2{\bf a}_2+\dots+x_n{\bf a}_n\,.$$ So, if ${\bf a}_i$'s are linearly independent, that means that $A{\bf x}=0\implies {\bf x}=0$.

$\endgroup$
1
  • $\begingroup$ Could you check my answer, please? $\endgroup$ Commented Dec 19, 2013 at 23:44
0
$\begingroup$

I think I have a counterexample to your professor's claim. Let A= $ \left( \begin{matrix} 0 & 1 \\ -1 & 0 \\ \end{matrix} \right)$; clearly, $A$ is linearly independent. And let $x^T=(1 \;\; 2)$. $$x^T A x=$$ $$\left(1 \;\; 2\right)\left( \begin{matrix} 0 & 1 \\ -1 & 0 \\ \end{matrix} \right)\left( \begin{matrix} 1\\ 2\\ \end{matrix} \right)=$$ $$\left(1 \;\; 2\right)\left( \begin{matrix} 2\\ -1\\ \end{matrix} \right)=0.$$ Where am I wrong here?

$\endgroup$
2
  • $\begingroup$ OP changed the question. This doesn't answer the question anymore. $\endgroup$ Commented Dec 20, 2013 at 0:01
  • 2
    $\begingroup$ It is about $x^T{\bf A}^TAx$ not $x^TAx$. $\endgroup$
    – Berci
    Commented Dec 20, 2013 at 0:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .