How is $I-vv^T$ the projection onto the orthogonal complement of the line through the origin in the direction $v$? Given a vector $v\in \mathbb{R}^n$, I have seen from several sources that $I-vv^T$ is the projection onto the orthogonal complement of the line through the origin in the direction $v$?
I have unfortunately memorized this expression instead of figuring out what it actually means.  I know that the orthogonal complement of a line is a plane with the line being its normal vector.  And from what I understand this means that $P=I-vv^T$ is a projection matrix which places vectors into that aforementioned plane.  But beyond that, I'm having difficulty connecting $I-vv^T$ to all vectors which are orthogonal to $v$ (which is what an orthogonal complement is, after all).
Can someone help me understand how this is true?
 A: Given a vector $x$, and a unit vector $v$, the projection of $x$ onto $v$ is going to be $(v^Tx)v=v(v^Tx)=(vv^T)x$.
Hence the orthogonal complement is $x-(v^Tx)v=x-(vv^T)x=(I-vv^T)x.$ 
A: Let $v$ an unit vector and $(v,u_1,\dots,u_{n-1})$ an orthonormal basis of $\mathbb{R}^n$ (Constructed by Grahm-Schmitt for example).
Then $x$ can be decomposed in that basis, i.e. : $$x=\lambda v+\sum_{i=1}^{n-1} \lambda_i u_i.$$
Then $v^Tx$ is the inner product and its values is $\lambda$. Thus $vv^Tx= \lambda v$. In other words, $vv^Tx$ is the orthogonal projection into the vector space spanned by $v$.
Then $$(I-vv^T)x=\sum_{i=1}^{n-1} \lambda_i u_i.$$
What we have done is removing the $v$ component of $x$. Since our basis is orthogonal, $$v^T(I-vv^T)x=\sum_{i=1}^{n-1}\lambda_i v^Tu_i=0.$$
$(I-vv^T)$ fails to be a projection into $ Span(u_1,..,u_{n-1})$ if $v$ is not a unit vector, as $vv^Tx=v\|v\|_2\lambda$, hence :
$$(I-vv^T)x=(1-\|v\|_2)\lambda v +\sum_{i=1}^{n-1} \lambda_i u_i.$$
which still have a component in $v$.
A: We have that
$$(vv^T)b=(v^Tb)v$$
is the othogonal projection of $b$ onto $v$ then
$$b-(vv^T)b=(I-vv^T)b$$
is the projection onto the orthogonal complement.
A: Let $E$ be the orthogonal space to the line defined by $v$. The orthogonal projection $P: \mathbb R^n \to E$ is the unique operator such that the following conditions are satisfied:


*

*The image of $P$ is $E$

*$\langle x - Px, E \rangle = 0$

*$E$ is $P$-invariant


To see uniqueness, suppose that $P'$ was another orthogonal projection, then
$$
0 = \langle Px - P'Px, E \rangle = \langle Px - P'x, E \rangle,
$$
and since $P, P'$ both map to $E$, they are equal. (Existence is typically shown by using an orthonormal basis of $E$ that is extended to an orthonormal basis of $\mathbb R^n$, using e.g. basis extension and Gram--Schmidt.) Hence, it is sufficient to show that your projection satisfies these properties.
So, let $x \in \mathbb R^n$.


*

*$I - v v^t$ maps to $E$, since $\langle (I - vv^t)x, v \rangle = 0$ (here we need $v$ to be a unit vector and use $\langle v v^t x, v \rangle = \langle x, v v^t v \rangle$. For reasons of dimension ($vv^t$ is a matrix of rank $1$), the image equals $E$.

*This is because $\langle -v v^t x, y \rangle = 0$ for $y \in E$.

*This is again by orthogonality, namely $v v^t y = v \langle v, y\rangle = 0$ for $y \in E$.

