# Stuck in derivation of PCA

I'm currently studying principal component analysis (PCA) from this lecture notes. I understand that we are trying to find the axis on which the variance of projection of all the data points is maximum.

Now where I'm stuck is in the formulation of PCA. In the above notes, on page number 5, our objective function is given as following.

$$\frac{1}{m}\sum_{i=1}^{m}(x^{(i)^{T}}u)^2 = \frac{1}{m}\sum_{i=1}^{m}u^Tx^{(i)}x^{(i)^T}u$$

How this is derived? I can't find any explanation on this step anywhere. As per my understanding shouldn't it be:

$$\frac{1}{m}\sum_{i=1}^{m}(x^{(i)^{T}}u)^2 = \frac{1}{m}\sum_{i=1}^{m}x^{(i)^T}ux^{(i)^T}u$$

Also in this question, the objective function is very different then the one mentioned above in the notes. (i.e from here and here it seems to be the distance between data point $$x^{(i)}$$ and the axis $$w$$, but in the notes and video lectures, it's mentioned that it's distance between projection of point and the origin).

What am I missing here? Any help would be appreciated

## 1 Answer

Note that ${x^{(i)}}^Tu$ is a scalar, since $x^{(i)}$ and $u$ are 2 vectors in $\Bbb R^n$. If I denote this number $\lambda$, then it corresponds to a $1\times 1$ matrix $[\lambda]$ and for this matrix, we have: $$[\lambda]^T [\lambda] = [\lambda]^2 = [\lambda^2]$$

To conclude, note that $[\lambda]^T[\lambda] = ({x^{(i)}}^Tu)^T ({x^{(i)}}^Tu) = u^T x^{(i)}{x^{(i)}}^Tu$.

• $x^{(i)^T}u$ is a dot product right? Jul 10 '18 at 17:55
• You're perfectly right.
– paf
Jul 10 '18 at 18:00
• Okay that's right. But how about my last paragraph of the question? Which is the correct representation of this formula? It's distance between origin and projection or distance between data point and the line? +1 for the explanation btw. Jul 10 '18 at 18:21
• Sorry for the delay. $x^{(i)^T}u$ is the length of the projection of $x^{(i)}$ on the line generated by $u$: mathworld.wolfram.com/DotProduct.html
– paf
Jul 12 '18 at 22:17