# Difference between the projection matrices arising from 1) the normal equations, and 2) an orthonormal basis of a subspace.

I'm having a bit of trouble tying two related ideas together, and I think I'm just missing a silly detail somewhere.

In the standard formulation of linear least-squares, it can be shown that given a full-rank matrix $$A$$, one can form the projection matrix $$P_A = A(A^TA)^{-1}A^T$$

which acts on vectors by orthogonally projecting them onto the column space of $$A$$.

On the other hand, given a subspace $$S \subseteq V$$ of a vector space, where $$S = \mathrm{Span}(\mathbf u_1, \mathbf u_2, \cdots \mathbf u_k)$$ and the $$\mathbf u_i$$ form an orthonormal basis of $$S$$, then one can construct a matrix with each $$\mathbf u_i$$ in the $$i$$th column: $$U = [\mathbf u_1, \mathbf u_2, \cdots \mathbf u_k].$$

One can then form the projection $$P_S = UU^T$$ which projects vectors in $$V$$ onto the subspace $$S$$.

I have two questions:

1. The matrix $$U$$ seems to be orthogonal, which would seem to mean it satisfies $$UU^T = I$$. What exactly am I missing?

2. The column space of $$U$$ is $$S$$ by construction, so how would one recover the formula for $$P_S$$ by letting $$A=U$$ in the formula for $$P_A$$? If $$U$$ is orthogonal, I see how $$(A^TA)^{-1} = I$$, but by the first question, it would also seem to reduce the remaining $$AA^T$$ to $$I$$ as well.

If $$k$$ is less than the dimension of $$V$$, then while $$U^T U$$ will be an identity matrix (because the columns of $$U$$ are orthonormal), $$UU^T$$ will generally not be an identity matrix (because there will be no guarantee that the rows of $$U$$ are orthonormal. In fact, you can show that if $$U$$ has size $$n\times k$$ where $$n > k$$, then $$UU^T$$ can never be the identity matrix.)
For example, suppose $$U = \begin{bmatrix} 1/\sqrt{2} & 0 \\ 1/\sqrt{2} & 0\\ 0 & 1\end{bmatrix}.$$
Then $$U^TU = I$$, while \begin{align*} UU^T &= \begin{bmatrix} 1/\sqrt{2} & 0 \\ 1/\sqrt{2} & 0\\ 0 & 1\end{bmatrix}\begin{bmatrix} 1/\sqrt{2} & 1/\sqrt{2} & 0\\ 0 & 0 & 1\end{bmatrix} \\ &= \begin{bmatrix}1/2 & 1/2 & 0\\ 1/2 & 1/2 & 0 \\ 0 & 0& 1\end{bmatrix}. \end{align*}