# Writing projection in terms of projection matrix

I've been reading this document. The goal is to find projection of $$b$$ on the line $$L$$ which is determined by vector $$a$$. ($$Proj_L(b)$$)

In the document, It is mentioned that If $$p$$ is thought as approximation to $$b$$, then $$e=b-p$$ is the error in that approximation (the word approximation is little confusing though, Isn't $$e$$ the exact value? Although this is not the problem).

Then we know that If $$p$$ lies on the line through $$a$$, then $$p = xa$$ for some $$x$$. We also know that, $$p$$ is orthogonal to $$e$$, therefore their dot product equates to zero:

$$a^T(b-xa)=0$$

$$a^Tb - a^Txa = 0$$

$$xa^Ta = a^Tb$$

$$x = \frac{a^Tb}{a^Ta}$$

Solving for $$p$$:

$$p = ax = a\frac{a^Tb}{xa^Ta}$$

First part was almost completely understandable, but in the second part this projection is written in the terms of projection matrix ("$$P: p = Pb$$"):

$$p = xa = \frac{aa^Ta}{a^Ta}$$

where did $$b$$ go? $$x$$ and $$a$$ have changed places, but isn't dot product commutative?

Then, $$P$$ is solved:

$$P = \frac{aa^T}{a^Ta}$$

Somehow, $$aa^T$$ is 3x3 matrix.

How was this concluded? From my knowledge, The general definition of projection matrix is $$A (A^{T}A)^{-1} A^T \vec{x}$$ (where $$A$$ is matrix)

Does the definition above has any relations with the projection matrix that was represented in document? If not, how was it derived?

Thank you!

• here $x = \frac{a^Tb}{xa^Ta}$ you have an extra x at the denominator, the issues for be are simply typo or change of symbols
– user
Mar 31, 2018 at 19:08
• To see the connection, try computing $A(A^TA)^{-1}A^T$ when $A$ is $n\times1$.
– amd
Mar 31, 2018 at 19:39

The key point is that from here

$$p = ax = a\frac{a^Tb}{a^Ta}$$

we can write in matrix form

$$p = ax = a\frac{a^Tb}{a^Ta}=\frac{aa^T}{a^Ta}b=Pb$$

From here we can generalize for a projection onto a subspace spanned by multiple vectors $a_i$.

Let consider the matrix $A=[a_1 \, a_2\,...\, a_n]$ and the vector $b$ to project then consider

$$Ax=p$$

the error is $e=b-p=b-Ax$ and it is miminized when $e$ is orthogonal to $Col(A)$ that is

$$A^Te=A^T(b-Ax)=0\implies A^Tb=A^TAx\implies x=(A^TA)^{-1}A^Tb$$

and then

$$p=Ax=A(A^TA)^{-1}A^Tb=Pb\implies P=A(A^TA)^{-1}A^T$$

• So from my understanding both methods can be used to represent projection as projection matrix? Also apologies for my lack of understanding, but is the second method some specific generalization? Mar 31, 2018 at 19:59
• @ShellRox The first case is the particular case of projection on a line (subspace with dimension $=1$, the second case is more general and valid for projection onto subspace ith dimension $\ge 1$.
– user
Mar 31, 2018 at 20:04
• Also final question, I've tested these equations (substituting variables with vectors) and I'm unable to understand how is first equation matrix? Maybe these variables should be matrices and not vectors? Apr 1, 2018 at 11:31
• $aa^T$ is a matrix and $a^Ta$ is a scalar, try with $a=(1,0,0)$
– user
Apr 1, 2018 at 11:39
• Pb is the projected vector p, try with some numerical example
– user
Apr 1, 2018 at 11:49