0
$\begingroup$

In an econometrics note it says:

Suppose that the rank of $X \in \mathbb R^{N \times K}$ is $K$, so that $X'X$ is positive definite.

I am trying to convince myself that this is true. I know that a p.d. matrix is invertible. Moreover, if rank of $X$ is $k$ then $X'X$ is full rank and invertible. However, doesn't any arbitrary matrix $M = X'X$ positive definite by definition? Because for any vector $y$ we have $y^\top M y = y^\top X' X y = (Xy)^\top Xy \geq 0$.

So I am not sure if the claim is wrong or if I am confused.

$\endgroup$
1
$\begingroup$

The rank condition on $X$ implies $N\ge K $, which makes sense in the context of the econometric linear regression origin of the question, even if not explicitly stated by the OP.

Let $M=X'X$. Without the rank condition on $X$ the matrix $M$ is positive semi-definite. With the rank condition, $M$ is also positive definite. It should be clear that $a'Ma=a'(X'X)a=(Xa)'(Xa)=\|Xa\|^2\ge0$; what you need to show is that $a'Ma=0$ implies $a=0$. But if $a'Ma=0$ then $\|Xa\|^2=0$ and so $Xa=0$. By the rank $K$ condition on $X$, we know $a=0$. And, under the rank condition, $M$ is invertible.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Oh, thanks! So regardless of $rank(X) = k$, we know that $X'X$ is p.s.d. But the rank condition implies that it is also strictly positive definite. If I am not wrong, there exists some psd matrices with zero eigenvalue, so it may be non invertible, but if it is positive definite (proven by the rank condition) then it is for sure invertible. Is this a true conclusion? $\endgroup$ – independentvariable Dec 8 '19 at 21:21
  • 1
    $\begingroup$ Yes: if the rank of $X$ is $k$ then $X'X$ is invertible, with rank $k$. $\endgroup$ – kimchi lover Dec 8 '19 at 21:24
  • 1
    $\begingroup$ @independentvariable positive semidefinite implies all eigenvalues are nonnegative. Positive definite implies they are all positive, which implies invertible (as the determinant is nonzero). $\endgroup$ – Clement C. Dec 8 '19 at 21:32
  • $\begingroup$ Thanks both of you! $\endgroup$ – independentvariable Dec 9 '19 at 0:20
1
$\begingroup$

tl;dr: Positive semi-definite, yes. Positive definite, no.

You are right, every matrix $M=X'X$ is positive semi-definite. However, the claim here is positive definite, which is stronger:

  • Positive semi-definite: for all $y$, $y^T M y\geq 0.$
  • Positive definite: for all $y$, $y^T M y > 0.$

See. e.g., the Wikipedia article.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.