3
$\begingroup$

I've been reading this document. The goal is to find projection of $b$ on the line $L$ which is determined by vector $a$. ($Proj_L(b)$)

Figure 1 in document linked above.

In the document, It is mentioned that If $p$ is thought as approximation to $b$, then $e=b-p$ is the error in that approximation (the word approximation is little confusing though, Isn't $e$ the exact value? Although this is not the problem).

Then we know that If $p$ lies on the line through $a$, then $p = xa$ for some $x$. We also know that, $p$ is orthogonal to $e$, therefore their dot product equates to zero:

$a^T(b-xa)=0$

$a^Tb - a^Txa = 0$

$xa^Ta = a^Tb$

$x = \frac{a^Tb}{a^Ta}$

Solving for $p$:

$p = ax = a\frac{a^Tb}{xa^Ta}$


First part was almost completely understandable, but in the second part this projection is written in the terms of projection matrix ("$P: p = Pb$"):

$p = xa = \frac{aa^Ta}{a^Ta}$

where did $b$ go? $x$ and $a$ have changed places, but isn't dot product commutative?

Then, $P$ is solved:

$P = \frac{aa^T}{a^Ta}$

Somehow, $aa^T$ is 3x3 matrix.

How was this concluded? From my knowledge, The general definition of projection matrix is $A (A^{T}A)^{-1} A^T \vec{x}$ (where $A$ is matrix)

Does the definition above has any relations with the projection matrix that was represented in document? If not, how was it derived?

Thank you!

$\endgroup$
2
  • 1
    $\begingroup$ here $x = \frac{a^Tb}{xa^Ta}$ you have an extra x at the denominator, the issues for be are simply typo or change of symbols $\endgroup$
    – user
    Mar 31, 2018 at 19:08
  • 1
    $\begingroup$ To see the connection, try computing $A(A^TA)^{-1}A^T$ when $A$ is $n\times1$. $\endgroup$
    – amd
    Mar 31, 2018 at 19:39

1 Answer 1

5
$\begingroup$

The key point is that from here

$$p = ax = a\frac{a^Tb}{a^Ta}$$

we can write in matrix form

$$p = ax = a\frac{a^Tb}{a^Ta}=\frac{aa^T}{a^Ta}b=Pb$$

From here we can generalize for a projection onto a subspace spanned by multiple vectors $a_i$.

Let consider the matrix $A=[a_1 \, a_2\,...\, a_n]$ and the vector $b$ to project then consider

$$Ax=p$$

the error is $e=b-p=b-Ax$ and it is miminized when $e$ is orthogonal to $Col(A)$ that is

$$A^Te=A^T(b-Ax)=0\implies A^Tb=A^TAx\implies x=(A^TA)^{-1}A^Tb$$

and then

$$p=Ax=A(A^TA)^{-1}A^Tb=Pb\implies P=A(A^TA)^{-1}A^T$$

$\endgroup$
8
  • $\begingroup$ So from my understanding both methods can be used to represent projection as projection matrix? Also apologies for my lack of understanding, but is the second method some specific generalization? $\endgroup$
    – ShellRox
    Mar 31, 2018 at 19:59
  • 1
    $\begingroup$ @ShellRox The first case is the particular case of projection on a line (subspace with dimension $=1$, the second case is more general and valid for projection onto subspace ith dimension $\ge 1$. $\endgroup$
    – user
    Mar 31, 2018 at 20:04
  • $\begingroup$ Also final question, I've tested these equations (substituting variables with vectors) and I'm unable to understand how is first equation matrix? Maybe these variables should be matrices and not vectors? $\endgroup$
    – ShellRox
    Apr 1, 2018 at 11:31
  • 1
    $\begingroup$ $aa^T$ is a matrix and $a^Ta$ is a scalar, try with $a=(1,0,0)$ $\endgroup$
    – user
    Apr 1, 2018 at 11:39
  • 1
    $\begingroup$ Pb is the projected vector p, try with some numerical example $\endgroup$
    – user
    Apr 1, 2018 at 11:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .