7
$\begingroup$

Prove, for real $X$, that $\text{rank}(X^TX)=\text{rank}(X)$.

Could anyone please help me with this problem? I've tried to use full-rank factorization and rank-related theorems mentioned in my book but still failed to solve this. I am learning linear algebra by myself and my book has no solutions manual so I find it really hard to get used to solve linear algebra problems.

Many thanks in advance for your help!

$\endgroup$

3 Answers 3

7
$\begingroup$

Assume $u$ is in the null space of $\mathbf{X}^T\mathbf{X}$.
Then $\mathbf{X}^T\mathbf{X}u=0 \implies u^T\mathbf{X}^T\mathbf{X}u = u^T0=0$.

Then assume, $u$ is not in the null space of $\mathbf{X}$.
Then $\mathbf{X}u=v$, (notice $u^T\mathbf{X}^T = v^T$).

But $u^T\mathbf{X}^T\mathbf{X}u = 0 \implies v^Tv=0$, which implies $v=\mathbf{X}u=0$.

Thus, $null(\mathbf{X}^T\mathbf{X})\subset null(\mathbf{X})$.

Proving the reverse inclusion is trivial.

So, $null(\mathbf{X}^T\mathbf{X}) = null(\mathbf{X})$.

By the rank and nullity theorem (Nullity of A + Rank of A = Total number of columns in A),
rank$(\mathbf{X}^T\mathbf{X})$ = rank$(\mathbf{X})$.

$\endgroup$
1
  • $\begingroup$ Thank you so much for your help! I can now finally move on to other problems :). $\endgroup$
    – Sophil
    Jun 4, 2018 at 17:00
6
$\begingroup$

Consider the map $T: Im(X)\to Im(X^TX)$ given by $T(w)=X^Tw$. We show that $T$ is an isomorphism.

$T$ is obviously onto, since given any $v\in Im(X^TX)$, we have $v=X^TXv'=X^T(Xv')=T(Xv')$ for some $v'$.

It remains to check that $T$ is one-to-one.

Suppose $T(w)=0$. We have $w=Xv$ for some $v$, so $X^TXv=0$. In particular, $(X^TXv, v)=0$. Thus $(Xv, Xv)=0$, so by positivity o the inner product, we conclude that $Xv=w=0$ as desired.

$\endgroup$
4
  • $\begingroup$ Thank you so much for your answer! Unfortunately, I haven't come to the section that mentioned in your solution (map, isomorphism...) yet. I would come back to your solution later when I learn those materials. $\endgroup$
    – Sophil
    Jun 4, 2018 at 16:57
  • $\begingroup$ I don't understand this answer. Are you assuming X is square and has full rank? $\endgroup$
    – ThighCrush
    Sep 23, 2022 at 8:48
  • $\begingroup$ @ThighCrush no, there is no assumption about X $\endgroup$ Sep 23, 2022 at 11:59
  • $\begingroup$ My bad, I understand now $\endgroup$
    – ThighCrush
    Sep 26, 2022 at 8:05
0
$\begingroup$

Lord KK above provided a sufficient answer. I will provide another approach which requires the knowledge of spectral decomposition, singular value decomposition (SVD) and the following lemma.

Lemma: For a full rank matrix $\mathbf{A}_{s\times s}$ and any $\mathbf{B}_{s\times t}$, $rank(\mathbf{A}\mathbf{B}) = rank(\mathbf{B})$.

$$\begin{aligned} rank(\mathbf{X}^T\mathbf{X}) & = \text{number of non-zero eigenvalues of }\mathbf{X}^T\mathbf{X} \\ & = \text{number of non-zero singular values of }\mathbf{X} \\ & \overset{*}{=} rank(\mathbf{X}), \end{aligned}$$

where the final equality follows from the definitions of SVD and the lemma, $$\begin{aligned} rank(\mathbf{X}) & = rank(\mathbf{U} \mathbf{D} \mathbf{V}^T) \\ & = rank( \mathbf{D} \mathbf{V}^T) \ \ \ \ (\text{Lemma})\\ & = rank( \mathbf{D}) \ \ \ \ \ \ \ \ \ \ (\text{Lemma})\\ & = \text{number of non-zero singular values of }\mathbf{X} \\ \end{aligned}$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .