# In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n u^T_k x \space u_k$?

In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n (u^T_k x) \space u_k$?

Where $\{u_1,...,u_n\}$ is orthonormal basis and $||u||^2=u^T_i u_i=1 \forall i$.

Is this some standard vector projection?

• It is a standard result on orhonormal bases, assuming the $u_k$ are one such. – Adomas Baliuka Jul 2 '18 at 16:54
• @AdomasBaliuka Where can I see it? In order to recall why it's so. – mavavilj Jul 2 '18 at 16:57
• I found : Let $V$ be a finite-dimensional inner product space, and let $\{e_1,e_2,...,e_n\}$ be an orthonormal basis of $V$. Then for each vector $v \in V$ we have that $v=<v,e_1>e_1+<v,e_2>e_2+...+<v,en>e_n$. However, how does this transform into the one with vector transpose? – mavavilj Jul 2 '18 at 17:00
• $\langle v, e_1 \rangle e_1 = \langle e_1, v \rangle e_1 = (e_1^\top v) e_1$ – angryavian Jul 2 '18 at 17:01

If $(u_1,\dots,u_n)$ is an orthonormal basis of any vector space $V$ equipped with an inner product $\langle \dot{}, \dot{} \rangle$ then
$$\forall x \in V,\; x =\sum_{k=1}^n \langle u_k, x\rangle u_k.$$
Indeed, if $a_1,\dots,a_n$ are the coordinates of $x$ in this basis, then $\displaystyle x=\sum_{j=1}^n a_j u_j$ and $\forall k\in \{1,\dots,n\},\;\langle x,u_k\rangle = \sum_{j=1}^n a_j \langle u_j,u_k\rangle = a_k$ since $\langle u_j,u_k\rangle = 1$ if $i=j$ and $0$ otherwise.
To conclude, note that on $\Bbb R^n$, the map defined by: $$\forall x\in\Bbb R^n,\;\forall y\in\Bbb R^n,\;\langle x,y\rangle := x^Ty$$ Is an inner product on $\Bbb R^n$.
• BTW, p.9 of this (math.ust.hk/~mabfchen/Math111/Week13-14.pdf) gives a proof for the orthogonal projection (into subspace $W \subset V$). However, how does it extend to all $V$? – mavavilj Jul 3 '18 at 13:11
• Take simply $W=V$: it's also a subspace of $V$ and in this case, the orthogonal projection onto $W=V$ is simply $\text{Id}_V$. – paf Jul 3 '18 at 16:26