# Difference between projection matrix and vector projection

I am trying to resolve my confusion regarding vector projection and projected vectors with projection matrices.

Suppose the column vectors of $$\mathbf{G}$$ are orthonormal. To my understanding the vector projection of vector $$\mathbf{a}$$ onto vector $$\mathbf{b}$$ is $$\hat{\mathbf{a}}(\frac{\mathbf{a}\cdot\mathbf{b}}{||\mathbf{a}||||\mathbf{b}||}$$). Then how come in, say, PCA the projection of a data matrix $$\mathbf{X}$$ onto the new orthonormal basis vectors of $$\mathbf{G}$$ is given by $$\mathbf{G}^T(\mathbf{X} - \mathbb{E}(\mathbf{X}))$$? Specifically, the first column vector of $$\mathbf{X}$$ ends up being $$\mathbf{G}^T(\mathbf{x}_1 - \mathbb{E}(\mathbf{x}_1)) = \sum_i\mathbf{g}_i^T(\mathbf{x}_1 - \mathbb{E}(\mathbf{x}_1))$$, where the $$i$$th component of the projected version of $$(\mathbf{x}_1 - \mathbb{E}(\mathbf{x}_1)$$ is given by the dot product with the $$i$$th column vector of $$\mathbf{G}$$. But at what point are we dividing with the length of the vector $$(\mathbf{x}_1 - \mathbb{E}(\mathbf{x}_1)$$ like in the vector projection? Or is there a reason why this is not necessary?

• The projection onto $b$ should be in the subspace generated by $b$. The projection formula you've provided doesn't meet that criteria. I think you're confusing the formula for $\cos \theta$ with the projection formula. Mar 6 '21 at 18:48

Suppose that $$\mathbf G$$ has orthonormal columns $$\mathbf g_1,\dots,\mathbf g_k$$. Because the columns are orthonormal, we can get the projection onto the span of these vectors by adding up individual projections. That is, the projection a vector $$\mathbf a$$ is given by $$\operatorname{proj}_{\mathbf G}(\mathbf a) = \frac{\mathbf g_1\cdot \mathbf a}{\|\mathbf g_1\|^2} \mathbf g_1 + \cdots + \frac{\mathbf g_k \cdot \mathbf a}{\|\mathbf g_k\|^2}\mathbf g_k \\ = (\mathbf g_1 \cdot \mathbf a)\mathbf g_1 + \cdots + (\mathbf g_k \cdot \mathbf a)\mathbf g_k.$$ The coefficients of this sum are $$\mathbf g_1\cdot \mathbf a, \dots, \mathbf g_k \cdot \mathbf a$$, which are the entries of the product $$\mathbf G^T\mathbf a.$$ To get the corresponding projected vector, take the corresponding linear combination of the columns of $$\mathbf G$$. In other words, we have $$\operatorname{proj}_{\mathbf G}(\mathbf a) = \mathbf G(\mathbf G^T \mathbf a).$$ With that in mind, we can find the coefficients of the projection for each column of the matrix $$\mathbf M = \mathbf X - \mathbb E(\mathbf X)$$ by computing $$\mathbf G^T\mathbf M$$ and obtain the vectors corresponding to these columns of coefficients with the product $$\mathbf G\mathbf G^T\mathbf M$$.