Let $u_1,...,u_k$ be an orthonormal basis for the subspace $W \subset \mathbb{R^m}$. Let $A = (u_1u_2...u_k)$ be the $m$ x $ k$ matrix whose columns are the orthonormal basis vectors, and define $P=AA^T$ to be the corresponding projection matrix.

a.) Given $v \in \mathbb{R^n}$, prove that its orthogonal projection $w \in W$ is given by matrix multiplication $w=Pv$

My attempt:

I know that if I let $u_1,...,u_n$ be an orthonormal basis for the subspace $W \subset V$. Then the orthogonal projection of a vector $v \in V$ onto W is $w = c_1u_1 +...+c_nu_n$ where $c_i=\langle v,u_i \rangle$ but I do not know how to show that this is given by $w=Pv$, where $W \subset \mathbb{R^m}$ and $v\in \mathbb{R^n}$?


1 Answer 1


First of all, note that $A$ is a kind of semi orthogonal matrix and its pseudo inverse is given by $A^{\dagger}=A^{T}$. Now you are trying to find a vector which is the orthogonal projection of $v$ in the space spanned by columns of $A$. This is also the closest vector to $v$ in the euclidean norm sense. Let us call it $v_{p}$. Clearly $v_{p}=Au$ for some vector $u$ in $\mathbb{R}^{m}$. So you are trying to solve \begin{align} \min_{u ~\epsilon~ \mathbb{R}^{m} } ||v-Au||^{2} \end{align} The solution for this is clearly $u=A^{\dagger}v=A^{T}v$. Substituting this for $v_p$, we obtain $v_p=AA^{T}v=Pv$.


Note that, we want to solve for a $u$ such that $v=Au$ (which is equivalent to that minimization problem). Now $A^{T}A=I$, since it is orthogonal. So multiplying by $A^{T}$ on both sides of $v=Au$ gives $u=A^{T}v$, and hence the required solution.

  • $\begingroup$ Thank you very much, but I have not been taught yet of psuedo inverse. Is there another way to prove this without using that? $\endgroup$
    – diimension
    Nov 3, 2012 at 22:29
  • $\begingroup$ @diimension Please see the revised solution. $\endgroup$ Nov 4, 2012 at 4:16
  • $\begingroup$ Thank you very much, Dineshdileep! $\endgroup$
    – diimension
    Nov 4, 2012 at 5:26
  • $\begingroup$ Dineshdileep, sry to bother again on this post but I did not understand your edited proof. For ex. how where did $v=Au$, does $u$ correspond to $w$ come from Can you, if you don't mind, add more detail? $\endgroup$
    – diimension
    Nov 17, 2012 at 6:11
  • $\begingroup$ $w$ corresponds to $Au$ and not $u$. $w$ is the vector in the $span(A)$ (column span) which is closest to $v$ as possible. So for all $u \epsilon \mathbb{R}^{N}$, we find a $w=Au$ such that it is closest to $v$. Putting that equality constraint makes sure that $w$ lies in span(A) $\endgroup$ Nov 18, 2012 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.