I have been trying to understand the theory behind the projection of a vector on a orthonormal bases :-

Suppose {$u_1,u_2,u_3,.......u_n$} be an orthonormal basis for a subspace $W$

Then the $\displaystyle proj_{_W} y=(y.u_1)u_1+(y.u_2)u_2+(y.u_3)u_3........(y.u_n)u_n$

But Here is my question :-If $U=[u_1,u_2.....u_n]$.

Then the $\displaystyle proj_{_W} y=U^TUy$

Since {$u_1,u_2,u_3,.......u_n$} is an orthonormal set, U is an orthogonal matrix

So, $U^TU=I\Rightarrow U^TUy=y$.

Where is the change happening?

  • $\begingroup$ Where does the equality $\operatorname{proj}_Wy=U^TUy$ come from. $\endgroup$ – José Carlos Santos May 20 '18 at 6:58
  • $\begingroup$ @JoséCarlosSantos that is not mentioned in the book(david C lay). $\endgroup$ – DRPR May 20 '18 at 7:40

Recall that in general the projection matrix is given by


and for U=A orthonormal we have


that is precisely

$$\displaystyle proj_{_W} y=(y.u_1)u_1+(y.u_2)u_2+(y.u_3)u_3........(y.u_n)u_n$$

| cite | improve this answer | |
  • $\begingroup$ Is there any proof/geometric interpretation of the projection matrix that you used? $\endgroup$ – DRPR May 20 '18 at 7:40
  • $\begingroup$ Refer to math.stackexchange.com/q/2717979/505767 $\endgroup$ – user May 20 '18 at 7:51
  • $\begingroup$ And also to en.m.wikipedia.org/wiki/… $\endgroup$ – user May 20 '18 at 7:53
  • $\begingroup$ thanks. understood $\endgroup$ – DRPR May 20 '18 at 7:57
  • $\begingroup$ You are welcome! Bye $\endgroup$ – user May 20 '18 at 7:58

Your reasoning is mostly correct, but you’ve applied it to the wrong expression. The correct expression for the projection is $UU^Ty$. What you have, $U^TUy$ doesn’t even make sense unless $W$ is the entire parent space: $U^TU$ is an $n\times n$ matrix, but if $W$ is a proper subspace, then $n$ is strictly less than the number of elements in $y$, so doesn’t have the right shape for their product to be defined.

The columns of $U$ are orthonormal, so $U^TU$ is indeed equal to $I_n$, but when $W$ is a proper subspace, $U$ is not square, therefore not orthogonal, so we can’t really say anything about $UU^T$ apart from its size. On the other hand, if $W$ is equal to the entire parent space, then $U$ is orthogonal and $UU^T=I$, but in that case the projection onto $W$ is just the identity map, anyway.

The formula $UU^Ty$ is just the original sum of individual projections onto the basis vectors that you started with, packaged up into matrix form: $$\sum_{i=1}^n (y\cdot u_i)u_i = \sum_{i=1}^n (u_i^Ty)u_i = U \begin{bmatrix}u_1^Ty \\ u_2^Ty \\ \vdots \\ u_n^Ty\end{bmatrix} = UU^Ty.$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.