In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n (u^T_k x) \space u_k$?

Where $\{u_1,...,u_n\}$ is orthonormal basis and $||u||^2=u^T_i u_i=1 \forall i$.

Is this some standard vector projection?

  • $\begingroup$ It is a standard result on orhonormal bases, assuming the $u_k$ are one such. $\endgroup$ – Adomas Baliuka Jul 2 '18 at 16:54
  • $\begingroup$ @AdomasBaliuka Where can I see it? In order to recall why it's so. $\endgroup$ – mavavilj Jul 2 '18 at 16:57
  • $\begingroup$ I found : Let $V$ be a finite-dimensional inner product space, and let $\{e_1,e_2,...,e_n\}$ be an orthonormal basis of $V$. Then for each vector $v \in V$ we have that $v=<v,e_1>e_1+<v,e_2>e_2+...+<v,en>e_n$. However, how does this transform into the one with vector transpose? $\endgroup$ – mavavilj Jul 2 '18 at 17:00
  • $\begingroup$ $\langle v, e_1 \rangle e_1 = \langle e_1, v \rangle e_1 = (e_1^\top v) e_1$ $\endgroup$ – angryavian Jul 2 '18 at 17:01

If $(u_1,\dots,u_n)$ is an orthonormal basis of any vector space $V$ equipped with an inner product $\langle \dot{}, \dot{} \rangle$ then

$$\forall x \in V,\; x =\sum_{k=1}^n \langle u_k, x\rangle u_k.$$

Indeed, if $a_1,\dots,a_n$ are the coordinates of $x$ in this basis, then $\displaystyle x=\sum_{j=1}^n a_j u_j$ and $\forall k\in \{1,\dots,n\},\;\langle x,u_k\rangle = \sum_{j=1}^n a_j \langle u_j,u_k\rangle = a_k$ since $\langle u_j,u_k\rangle = 1$ if $i=j$ and $0$ otherwise.

To conclude, note that on $\Bbb R^n$, the map defined by: $$\forall x\in\Bbb R^n,\;\forall y\in\Bbb R^n,\;\langle x,y\rangle := x^Ty$$ Is an inner product on $\Bbb R^n$.

  • $\begingroup$ BTW, p.9 of this (math.ust.hk/~mabfchen/Math111/Week13-14.pdf) gives a proof for the orthogonal projection (into subspace $W \subset V$). However, how does it extend to all $V$? $\endgroup$ – mavavilj Jul 3 '18 at 13:11
  • $\begingroup$ Take simply $W=V$: it's also a subspace of $V$ and in this case, the orthogonal projection onto $W=V$ is simply $\text{Id}_V$. $\endgroup$ – paf Jul 3 '18 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.