Projection Matrix Formulae Comparison and Intuition I wanted some intuition behind the formulae of projection of point to a subspace. Particularly I wanted to compare it to the situation where the subspace is just a 1D line. 
Let $b$ be the point to be projected. 
For 1 dimensional subspace projection matrix $P=\frac{a a^{\mathrm{T}}}{a^{\mathrm{T}} a}$ , so $proj=\frac{a a^{\mathrm{T}}}{a^{\mathrm{T}} a}b$
For $N$ dimensional subspace the extension of the projection formulae is :
$P=A\left(A^{\mathrm{T}} A\right)^{-1} A^{\mathrm{T}}$
Many similar elements can be seen as said by my book. Instead of $a^{\mathrm{T}} a$ in the denominator we have $(A^{\mathrm{T}} A)^{-1}$ and the individual $a$ and $a^T$ are present as $A$ and $A^T$ as well. But why is $(A^{\mathrm{T}} A)^{-1}$  a sutiable replacement for the normalization that $a^{\mathrm{T}} a$ does? It seems to me like a very different matrix than simply normalization action of $a^{\mathrm{T}} a$. How is $A^T A$ similar in more than 1 dimensions? $A^T A$ is dot product of every column of A with every other so I'm sure something is going on.
The order of the operations is also more important than in the 1D case. Someone told me some argument relating to change of basis but I don't really see it. I understand the proof very well but want some insight into this formulae.
 A: It’s instructive to review one of the ways to derive the formula for orthogonal projection onto a vector $a$. By definition, if $\mathbf\pi_a v$ is the orthogonal projection of $v$ onto $a$, then $v-\mathbf\pi_av$ is orthogonal to $a$. Now, $\mathbf\pi_av=ka$ for some scalar $k$, so we have the condition $$a^T(v-ka) = 0,$$ from which $$k = {a^Tv\over a^Ta}$$ and so $$\mathbf\pi_a v = ka = {a^Tv\over a^Ta}a = {aa^T\over a^Ta}v.$$ 
We can proceed in a similar fashion for orthogonal projection onto the column space of $A$. This time, we want a linear combination of the columns of $A$, which we can write as $Aw$. Note also that the elements of $A^Tv$ are the dot products of the columns of $A$ with $v$. Proceeding as above, we then have $$A^T(v-Aw)=0 \\ A^TAw = A^Tv.$$ Now, if the columns of $A$ are linearly independent, then it turns out that $A^TA$ is invertible (prove this!) and we can continue with $$w = (A^TA)^{-1}A^Tv \\ \therefore \mathbf\pi_Av = Aw = A(A^TA)^{-1}A^Tv.$$ If $A$ doesn’t have full column rank, then you need to use a pseudoinverse, as noted in a comment to your question. 
At this point it’s helpful to examine what happens when the columns of $A$ form an orthonormal set. In that case, $A^TA$ is the identity matrix, so the formula reduces to $AA^Tv$, but this expands into $\sum(a_j^Tv)a_j$, i.e., into the sum of the individual projections onto the columns of $A$. When the $a_i$ are pairwise orthogonal, but not necessarily unit vectors, $A^TA$ is diagonal with its diagonal elements being $a_i^Ta_i$, so we again have the sum of the individual projections onto the columns of $A$.  
It works out this neatly because when the $a_i$ are orthogonal, there’s no “cross-talk:” if you add a multiple of $a_i$ to $v$, this has no effect on the value of the dot product of $v$ with any of the other columns of $A$. On the other hand, if they’re not orthogonal, changing the component of $v$ in the direction of $a_i$ can also have an effect on its components in other directions given by the other columns. The Gram matrix $A^TA$ captures these interactions among the columns of $A$ (its elements are their pairwise dot products) and multiplying by the inverse of this matrix magically (to me, at any rate) untangles all of those interactions.
