# Show that $R\!\left(X^T\right)\subseteq R\!\left(X^TX\right).$

I need to show that if $$c\in R\!\left(X^T\right),$$ then $$c\in R\!\left(X^T X\right).$$ Here, assume that $$X$$ is a finite matrix, not necessarily square, and that $$c$$ is a vector of the appropriate shape to make the matrix multiplications valid. Also, for notation: $$R\!\left(X^T\right)$$ is the range space of $$X^T,$$ which in turn is the transpose matrix of $$X.$$ Assume everything in sight is real, not complex.

I know that $$c\in R\!\left(X^T\right)$$ if and only if there exists $$z$$ s.t. $$X^Tz=c.$$ Also, $$c\in R\!\left(X^T X\right)$$ if and only if there exists a vector $$\lambda$$ such that $$c=X^TX\lambda.$$ So it appears, somehow, that I must show $$z=X\lambda.$$ It doesn't seem obvious to me that $$z$$ should be in the range of $$X,$$ which is essentially what that's saying.

Important note: I just realized that if $$X$$ is not square, then we have the possible objection that the dimensions don't work out. That is, the number of components in a vector in $$R\!\left(X^T\right)$$ is not necessarily the same as the number of components in a vector in $$R\!\left(X^TX\right).$$ (Notice I'm not using the term "dimension" here - that's deliberate.) That's certainly true, but let's assume that this is not an objection - which, for all I know, might be tantamount to saying that $$X$$ is square. Or maybe we can simply consider that $$X\lambda$$ can be considered equal to $$z$$ if we can lop off enough components from one or the other such that what remains is equal.

How do I continue?

• I don't understand, your argument in the second paragraph is correct. What did you think was wrong with it? Dimensions are also not an issue: Just take $X$'s dimensions to be $n \times p$ and reason about dimensions of the products, you will see they work out Jan 13, 2022 at 18:37
• @0XLR Well, it's not a complete argument at all. I've outlined, as far as I can see, what must be shown, but I have not shown it. Why should $z=X\lambda?$ Jan 13, 2022 at 18:45
• The statement is false. Consider $X^T=\pmatrix{1&i}$ for instance. Are you considering real matrices? Jan 13, 2022 at 19:03
• @user1551 Well, the question comes from the book Linear Models with R, a statistics textbook. As statistics is dealing with real numbers the vast majority of the time, that would be a safe assumption. I'll edit the question to reflect that. Thanks! Jan 13, 2022 at 19:07

This question has come up in various forms many times on this site. In general, we know that $$R(AB) \subset R(A)$$ for any matrices $$A$$ and $$B$$. However, in your case, you get the reverse inclusion by dimension considerations.

The claim is that $$R(X^\top) = R(X^\top X)$$ because you have the inclusion $$R(X^\top X)\subset R(X^\top)$$ and the two subspaces have the same dimension. To see this, we apply the nullity-rank theorem. We observe instead that $$N(X)\subset N(X^\top X)$$ (why?) and note that if $$X^\top Xv = 0$$, then $$0 = X^\top Xv\cdot v = Xv\cdot Xv = \|Xv\|^2$$, so $$Xv=0$$. This shows that $$N(X^\top X)\subset N(X)$$ and therefore that $$N(X^\top X) = N(X)$$. It follows from nullity-rank that $$\text{rank} (X^\top X) = \text{rank}(X)$$. Since $$\text{rank} (X^\top) = \text{rank}(X)$$, we have $$\text{rank}(X^\top) = \text{rank}(X^\top X)$$, so $$\dim(R(X^\top)) = \dim(R(X^\top X))$$, as required.

• Many thanks! I think you had my wife, Susan Keister nee Garrison in one of your geometry classes at UGA. Jan 13, 2022 at 19:34
• What goes wrong with the proof for the complex row vector $X = (1 \quad i)$? I am guessing it is "inner product" step where you argued $X^T X v \cdot v = \|Xv\|^2$? Jan 13, 2022 at 19:37
• @0XLR Yes, it's the inner product: $x^Tx = 0$ doesn't imply $x = 0$ for complex vectors. Jan 13, 2022 at 19:44
• @0XLR Right, with complex matrices you have to use the Hermitian inner product to do geometry. Jan 13, 2022 at 19:45

This might be helpful for future visitors:
$$col(\cdot) = range(\cdot)$$ denotes columnspace, $$row(\cdot)$$ denotes rowspace, $$N(\cdot)$$ denotes nullspace, $$\cdot^{\perp}$$ denotes orthogonal complement

consider arbitrary $$X \in \mathbb{R}^{n \times m}$$

Observation 1: $$row(X) = col(X^{T})$$
proof:
$$v \in row(X) \implies v = X^T \mu$$ for some $$\mu \in \mathbb{R}^{n} \implies v \in col(X^{T})$$
conversely, $$v \in col(X^{T}) \implies v = X^T \mu$$ for some $$\mu \in \mathbb{R}^{n} \implies v \in row(X)$$

Observation 2 $$N(X)^{\perp} = row(X)$$
proof:
proceed with $$N(X)^{\perp} = row(X) \iff N(X) = row(X)^{\perp}$$ [as $$(U^{\perp})^{\perp} = U$$]
$$v \in N(X) \implies Xv = 0 \implies X_i \cdot v = 0 \text{ for all rows } X_i \text{ of } X$$
therefore $$\mu \cdot v = 0, \forall \mu \in row(X)^{\perp} \implies v \in row(X)^{\perp}$$
converse follows.

Observation 3: $$N(X^TX) = N(X)$$
proof: $$v \in N(X^TX) \implies X^TXv = 0 \implies v^TX^TXv = 0 \implies ||Xv||_2^2 = 0 \implies Xv = 0 \implies v \in N(X)$$, conversely $$v \in N(X) \implies Xv = 0 \implies X^TXv = 0 \implies v \in N(X^TX)$$

Claim: $$col(X^TX) = col(X)$$
short proof:
$$N(X) = row(X)^{\perp} = col(X^T)^{\perp}$$
$$N(X^TX) = row(X^TX)^{\perp} = col(X^TX)^{\perp}$$ [as $$(X^TX)^T = X^TX$$]
$$N(X^TX) = N(X) \implies col(X^TX)^{\perp} = col(X^T)^{\perp} \implies col(X^TX) = col(X^T)$$

alternative:
$$v \in col(X^TX) \implies v = X^TX \mu = X^T(X \mu) \text{ for some \mu \in \mathbb{R}^m} \implies v \in col(X^T)$$

$$v \in col(X^T) \implies v \in row(X) \implies v \in N(X)^{\perp} \implies v \in N(X^TX)^{\perp}$$ \ $$\implies v \in row(X^TX) \implies v \in col(X^TX)$$ [as $$(X^TX)^T = X^TX$$]

$$range(X) \subseteq range(X^TX)$$ folows from claim