understanding projection matrices Let us look at two linearly independent vectors $v_1$ and $v_2$, which span a plane in $\mathbb{R}^n$ through the origin. Let us form a matrix $V$ out of these vectors so that the first column of $V$ is the vector $v_1$ and the second column is the vector $v_2$. Show that the the matrix $P=V(V^TV)^{-1}V^T$ projects all vectors $x\in\mathbb{R}^n$ orthogonally onto the plane spanned by $v_1$ and $v_2$.
 A: Hint: You need to show two things.


*

*$P\mathbf{x}$ is in the plane, i.e. $P\mathbf{x}\in\text{span}(\mathbf{v}_1,\mathbf{v}_2)$, for all $\mathbf{x}\in\mathbb{R}^n$

*$P\mathbf{x}$ (the projection) and $\mathbf{x}-P\mathbf{x}$ are perpendicular, i.e. $\langle \mathbf{x},\mathbf{x}-P\mathbf{x}\rangle=0$.


To help you get started on (1), the point is that we need to find scalars $c_1$ and $c_2$ so that $\mathbf{x}=c_1\mathbf{v}_1+c_2\mathbf{v}_2$. (That is what it means for $\mathbf{x}$ to lie in the plane spanned by $\mathbf{v}_1$ and $\mathbf{v}_2$.) But $$c_1\mathbf{v}_1+c_2\mathbf{v}_2=V\begin{pmatrix}c_1\\c_2\end{pmatrix}=V\mathbf{c}$$ where $\mathbf{c}$ is the vector $(c_1,c_2)\in\mathbb{R}^2$.
So we want to solve the equation $$V\mathbf{c}=\mathbf{x}$$ for $\mathbf{c}$. Now you have to be a little careful: the matrix $V$ is not square...
A: To show $(1)$, note that $V=[v_1\ v_2]$. Then
$$
[Pv_1\ Pv_2]=PV=V(V^TV)^{-1}V^TV=V=[v_1\ v_2].
$$
So $Pv_1=v_1$, $Pv_2=v_2$.
A: Let $\{ v_1, v_2, w_1, \dots, w_{n-2} \}$ be a basis for V. By Graham-Schmidt, we can choose the $w$'s to be orthogonal to the $v$'s. We can write any x in V as: 
$\textbf{x} = (a_1v_1 + a_2v_2) + (b_1w_1 + \dots + b_{n-2}w_{n-2})$.
Or in matrix notation:
$\textbf{x} = V \left( \begin{smallmatrix}
a_1 \\ a_2
\end{smallmatrix} \right) + W \left( \begin{smallmatrix}
w_1 \\ \vdots \\ w_{n-2}
\end{smallmatrix}\right)$
where W is $[w_1 \dots w_{n-2}]$.
Hence $P\textbf{x} = (PV) \left( \begin{smallmatrix}
a_1 \\ a_2
\end{smallmatrix} \right) + (PW) \left( \begin{smallmatrix}
w_1 \\ \vdots \\ w_{n-2}
\end{smallmatrix}\right)$
by linearity and associativity of matrix multiplication. But $PV = V$, and $V^TW$ is zero because it's formed by taking the dot products of v's with w's, which are orthogonal. This makes PW a zero matrix, as well. So
$P\textbf{x} = V \left( \begin{smallmatrix}
a_1 \\ a_2
\end{smallmatrix} \right) = a_1v_1 + a_2v_2$, which is the orthogonal projection.
A: I found this approach helpful so I would like to share it.
Useful results, if $P$ is a projection matrix then:


*

*$ P\circ P = P$ , i.e projector $P$ projects the kernel onto itself.

*$ Pv=v\ \forall\ v\in V$ where $V=\mathrm{null}(P)$, direct consequence of $1.$

*$ Pu=\emptyset\ \forall\ (u \mid u.v=\emptyset\ \forall v\in V) $ 

*$ P^{\dagger}=P $, iff $P$ represents an orthogonal projection.


using $(2)$ it is clear that $V$ is spanned by the eigenvectors of P which correspond to eigenvalue $\lambda=1$. Let $w1,w2$ be the eigenvectors for $\lambda=1$, we know that $W=(w1,w2)$ would diagonalize the matrix $P$. Thus,
$W^TPW=1$ or $P=WW^T$ this is what one would call an orthonormal basis. 
Now we want to change the basis from $W$ to $V$ giving $W=S^{-1}V$ and
$S=VV^T$ which would give you a good starting point.
One way to see it is to notice that $(V^TV)^{-1}$ is a kind of normalization factor. 
