I have learned that projecting a vector a onto a vector b is done by multiplying the orthogonal projection of a (say $\mathbf{a_o}$) with the unit vector $\mathbf{\hat{b}}$ in the direction of b =

  • Orthogonal projection of a onto b = $\mathbf{a_o}$ * $\mathbf{\hat{b}}$

However in the context of principal component analysis, I often read that the projection of the data D onto an eigenvector v is simply $\mathbf{v}^{T}$*$D$ (and so we don't take the orthogonal components of the data vectors of D..?) - why is that?


  • $\begingroup$ Do you know how to compute $\mathbf{a_0}$ from $\mathbf{a}$ and $\mathbf{b}$? $\endgroup$ – angryavian Dec 14 '15 at 21:54
  • $\begingroup$ I haven't any experience with PCA so please excuse my ignorance. Is $D$ a matrix, where each column represents a vector? $\endgroup$ – Alexander Heyes Dec 14 '15 at 21:56

For two vectors $a,b \in \mathbb R^n$, the orthogonal projection of $a$ onto the span of $b$ is given by $P_b(a) = \langle a ,b\rangle \frac{b}{\langle b,b\rangle}$, where the brackets denote the scalar product, i.e. $\langle a, b\rangle = a^Tb = b^Ta$. If $b$ has norm $1$, i.e. $\langle b,b\rangle =1,$ this simplifies to $P_b(a) = \langle a ,b\rangle b = (b^Ta)b$.

If you want to plot this vector, the vector part ($b$) tells you in which direction to go and the scalar factor ($\langle a ,b\rangle = (b^Ta)$) tells you how far. If you already know in which direction to go, the only interesting information is the distance.

The data matrix $D$ in your example consists of a lot of vectors like $a$, written down in columns.

So if you just consider a (normed) vector $b$ and look at $b^T D, $ this will be a 1xn column vector.

What does this vector tell us? The first entry is the scalar factor for the first column, the second for the second, and so on. What's missing is the vector $b$ to multiply it with.

That is: $b^T D$ is the column vector which contains the coordinates of the columns of $D$ with respect to $b$. If you want to plot this, you can forget about the actual direction of $b$, as the aim is to consider $b$ as the 'new x' and the orthogonal complement of $b$ as the new 'y'. We don't care about the direction, only the coordinates: If $c$ is orthogonal to $b$ and has norm 1, then we can calculate the coordinates of the columns of $D$ as $c^TD$. If we plot the first entry of $b^TD$ against the first entry of $c^TD$, we will have the the data point corresponding to the first column represented in the new coordinate system, and so on for all the other columns of $D$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy