In the following document about projection onto subspaces, the author is computing the transformation matrix to project a vector $b$ onto a line formed by vector $a$.

Illustration of the problem

Since the projected vector $p$ is on the $a$ line, therefore it can be expressed as $p = \bar{x}a$. The projection "line" which is the vector $b - p$ is orthogonal to $a$. This means the dot-product of these two is zero:

$$a \dot{} (b - p) = a^T(b - \bar{x}a) = 0$$

It comes to the conclusion that $\bar{x} = \frac{a^T b}{a^T a}$ so

$$p = ax = a \frac{a^T b}{a^T a}$$ (Which I fully understand)

Then the objective is to find the projection matrix P such as $p = P b $. The author wrote that $p = \bar{x}a = \frac{aa^Ta}{a^Ta}$ and so $P = \frac{aa^T}{a^Ta}$

How does he deduced this last part? I cannot see how he goes from $$p = \bar{x} a = \frac{a^T b}{a^T a} a$$ to $$p = \frac{aa^Ta}{a^Ta}$$

And even then, why would $P$ be $\frac{aa^T}{a^Ta}$ since there are absolutely no $b$ in this last expression to identify a $p = Pb$ ?


You can use this way to verify: Let $a=\binom{a_1}{a_2},b=\binom{b_1}{b_2}$. Then \begin{eqnarray} p&=&\bar{x}a=\frac{a^Tb}{a^Ta}a=\frac{a_1b_1+a_2b_2}{a^Ta}\binom{a_1}{a_2}\\ &=&\frac{1}{a^Ta}\binom{a_1(a_1b_1+a_2b_2)}{a_2(a_1b_1+a_2b_2)}\\ &=&\frac{1}{a^Ta}\left(\begin{matrix}a_1^2&a_1a_2\\ a_1a_2&a_2^2 \end{matrix}\right)\binom{b_1}{b_2}\\ &=&\frac{1}{a^Ta}aa^Tb \end{eqnarray} Thus $$ P=\frac{1}{a^Ta}aa^T. $$


We could derive it a bit more intuitive as follows :
Let's say we want to project the vector $x$ onto the direction of an given vector $a$.
1) The first thing we would do is to get the unit vector in the direction of $a$ which is $u={a\over {a^T a}}$
2) When we project the vector $x$ onto $a$ by taking $x.a$ it translates as $a^T x$.
3) So, $a^Tx$ gives the magnitude of the dot product of the two vectors.
4) But, the direction of the projected component is $u$.
5) In the end we have, $u(a^Tx)$.


I think there is a typo in the book.

What the author meant is, you want:

$$Pb = a\frac{a^{T}b}{a^{T}a}$$

Where the expression on the right is the $p$ that you found which you understand. This is the same as:

$$Pb = \left(\frac{aa^{T}}{a^{T}a}\right)b$$

And the form of $P$ follows.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.