# I - P projection matrix

Learning Linear Algebra on my own time. Came upon a problem, which asked to find a projection matrix $P$ onto a column space of $A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{bmatrix}$ and project vector $b = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4\end{bmatrix}$ onto it.

The solution if fairly straight forward and the answer is $P = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0\end{bmatrix}$ and $Pb = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 0\end{bmatrix}$.

There is another question to this problem.

For the matricies $A$ and $P$, if $P$ projects onto the columsn space of $A$ then what does $I-P$ projects onto?

Intuition told me that it is projecting onto left nullspace of $A$, i.e. onto the nullspace of $A^T$. And indeed it is not hard to demonstrate. Simply do the $I-P$ calculation and verify that it projects onto a left nullspace of $A$.

Here is my question. Say I have a generic matrix $A$ with its projection matrix $P$ that projects onto a column space of $A$. Is it always the case that $I-P$ will project onto a left nullspace of $A$? If it is, then how do I prove it? If it isn't, are there certain conditions that this will make it true? How do I find those conditions?

Yes, that is true in general.

First, note that by definition the left nullspace of $A$ is the orthogonal complement of its column space (which, by the way, is unique, and so we say "the column space of $A$" rather than "a column space"), because $A^T x = 0$ if and only if $x$ is orthogonal to every column of $A$.

Therefore, if $P$ is an orthogonal projector onto its column space, then $I - P$ is a projector onto its orthogonal complement, i.e., the nullspace of $A^T$. To see this, first note that, by definition, $Px = x$ for all $x$ is in the column space of $A$. Thus,

$(I - P)x = x - P x = x - x = 0$.

On the other hand, if $y$ is in the left nullspace of $A$, then $P y = 0$, and so

$(I - P)y = y - Py = y - 0 = y$.

Edit: also, if $P$ is an orthogonal projector, it is self-adjoint, and so is $I-P$, because the sum of two self-adjoint linear operators is also self-adjoint. Hence, in that case, $I-P$ is also an orthogonal projector.

• I guess I'm missing something. Here is that part that I don't understand: Therefore, if $P$ is an orthogonal projector onto its column space, then $I−P$ is a projector onto its orthogonal complement. How do you arrive to this conclusion? Nov 6 '17 at 19:36
• Note that $(I-P)x = 0$ if and only if $Px = x$, which means that $(I-P)x = 0$ if and only if $x$ is in the column space of $A$. Similarly, $(I-P)y = y$ if and only if $Py = 0$, i.e., $y$ is in the left nullspace of $A$. Hence, for an arbitrary vector $z$, we have $z = Pz + (I - P)z = x + y$, where $Pz = x$ lies in the column space of $A$ and $(I-P)z = y$ lies in the left null space $A$. Also since $P$ is a projector it must be idempotent, meaning $P^2 = P$, and so $(I-P)(I-P) = I - 2P + P^2 = I - 2P + P = I-P$, i.e., $(I-P)$ is also idempotent. The claim thus follows by definition. Nov 6 '17 at 20:00

Let's say that $$x$$ has been projected by $$P$$, is an image. So the $$Px = x$$. If I were to project this x by $$I-P$$, $$(I-P) x=x-P x=x-x=0$$. $$x$$ is itself in the projection of $$P$$, $$0$$ in the projection of $$I-P$$. That means that $$P$$ and $$I-P$$ are orthogonal complements.