1
$\begingroup$

Learning Linear Algebra on my own time. Came upon a problem, which asked to find a projection matrix $P$ onto a column space of $A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{bmatrix}$ and project vector $b = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4\end{bmatrix}$ onto it.

The solution if fairly straight forward and the answer is $P = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0\end{bmatrix}$ and $Pb = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 0\end{bmatrix}$.

There is another question to this problem.

For the matricies $A$ and $P$, if $P$ projects onto the columsn space of $A$ then what does $I-P$ projects onto?

Intuition told me that it is projecting onto left nullspace of $A$, i.e. onto the nullspace of $A^T$. And indeed it is not hard to demonstrate. Simply do the $I-P$ calculation and verify that it projects onto a left nullspace of $A$.

Here is my question. Say I have a generic matrix $A$ with its projection matrix $P$ that projects onto a column space of $A$. Is it always the case that $I-P$ will project onto a left nullspace of $A$? If it is, then how do I prove it? If it isn't, are there certain conditions that this will make it true? How do I find those conditions?

$\endgroup$
2
$\begingroup$

Yes, that is true in general.

First, note that by definition the left nullspace of $A$ is the orthogonal complement of its column space (which, by the way, is unique, and so we say "the column space of $A$" rather than "a column space"), because $A^T x = 0$ if and only if $x$ is orthogonal to every column of $A$.

Therefore, if $P$ is an orthogonal projector onto its column space, then $I - P$ is a projector onto its orthogonal complement, i.e., the nullspace of $A^T$. To see this, first note that, by definition, $Px = x$ for all $x$ is in the column space of $A$. Thus,

$(I - P)x = x - P x = x - x = 0$.

On the other hand, if $y$ is in the left nullspace of $A$, then $P y = 0$, and so

$(I - P)y = y - Py = y - 0 = y$.

Edit: also, if $P$ is an orthogonal projector, it is self-adjoint, and so is $I-P$, because the sum of two self-adjoint linear operators is also self-adjoint. Hence, in that case, $I-P$ is also an orthogonal projector.

$\endgroup$
2
  • $\begingroup$ I guess I'm missing something. Here is that part that I don't understand: Therefore, if $P$ is an orthogonal projector onto its column space, then $I−P$ is a projector onto its orthogonal complement. How do you arrive to this conclusion? $\endgroup$
    – flashburn
    Nov 6 '17 at 19:36
  • $\begingroup$ Note that $(I-P)x = 0$ if and only if $Px = x$, which means that $(I-P)x = 0$ if and only if $x$ is in the column space of $A$. Similarly, $(I-P)y = y$ if and only if $Py = 0$, i.e., $y$ is in the left nullspace of $A$. Hence, for an arbitrary vector $z$, we have $z = Pz + (I - P)z = x + y$, where $Pz = x$ lies in the column space of $A$ and $(I-P)z = y$ lies in the left null space $A$. Also since $P$ is a projector it must be idempotent, meaning $P^2 = P$, and so $(I-P)(I-P) = I - 2P + P^2 = I - 2P + P = I-P$, i.e., $(I-P)$ is also idempotent. The claim thus follows by definition. $\endgroup$ Nov 6 '17 at 20:00
0
$\begingroup$

Let's say that $x$ has been projected by $P$, is an image. So the $Px = x$. If I were to project this x by $I-P$, $$ (I-P) x=x-P x=x-x=0 $$. $x$ is itself in the projection of $P$, $0$ in the projection of $I-P$. That means that $P$ and $I-P$ are orthogonal complements.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.