# Relation between projection matrix and linear span

Let $X$ be an $m \times n$ matrix. Define the orthogonal projection onto the column space of $X$ as

$$P(X):=X(X'X)^{-1}X'$$

Also, define the linear span of a set of $m\times 1$ column vectors of $X$ as $\mbox{span}(X)$.

I'm wondering what the relation between $P(X)$ and vector space $\mbox{span}(X)$ is. Can you help me with an intuitive explanation? Intuitive and easy interpretation would be appreciated.

• I am not following some things. $X$ gives us $n$ vectors of $\mathbb{R}^m$. We define the projection on $\mathbb{R}^m$ ? and we define the projection over the span of the columns of $X$ ? So I suppose $n\leq m$ ? Another thing, I don't catch what means $X'$. What do you refer as a set of $mx1$ vectors, which vectors? – HFKy Jul 20 '18 at 2:07
• First, $X'$ is the transpose of $X$. It's assumed that $n \leq m$ holds. $m\times 1$ vectors refer to each $m\times 1$ columns of $X$. – user568810 Jul 20 '18 at 2:37
• $m$x$1$ columns ? $X$ doesn't have $n$ column vectors, and $n \leq m ?$ – HFKy Jul 20 '18 at 2:39
• Consider EACH column of $X$. $X$ has $n$ number of $m\times 1$ vectors. And yes, $n\leq m$. – user568810 Jul 20 '18 at 2:41
• "mx1 vectors" ???. $X$ is mxn is impossible to get "mx1 vectors" whatever it means if $n \leq m$. Being $X$ mxn you have $n$ column vectors of $\mathbb{K}^m$ meaning that each of these vectors have $m$ components. – HFKy Jul 20 '18 at 2:45

The columns of design matrix $X$ form a vector space if there is no intercept in the model $Y= X\beta$, in a case of an intercept this is an affine space. The intuitive relation is that the hat matrix $H = X(X'X)^{-1}X'$ projects the $n$ dimensional response vectors $y$ into the space that is spanned by your explanatory variables. Namely, $Hy=\hat{y}$ gives you the "closest" vector that can be uniquely represented by a linear combination of the columns of $X$ (explanatory variables).