Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $u_1,...,u_k$ be an orthonormal basis for the subspace $W \subset \mathbb{R^m}$. Let $A = (u_1u_2...u_k)$ be the $m$ x $ k$ matrix whose columns are the orthonormal basis vectors, and define $P=AA^T$ to be the corresponding projection matrix.

a.) Given $v \in \mathbb{R^n}$, prove that its orthogonal projection $w \in W$ is given by matrix multiplication $w=Pv$

My attempt:

I know that if I let $u_1,...,u_n$ be an orthonormal basis for the subspace $W \subset V$. Then the orthogonal projection of a vector $v \in V$ onto W is $w = c_1u_1 +...+c_nu_n$ where $c_i=\langle v,u_i \rangle$ but I do not know how to show that this is given by $w=Pv$, where $W \subset \mathbb{R^m}$ and $v\in \mathbb{R^n}$?

share|cite|improve this question
up vote 1 down vote accepted

First of all, note that $A$ is a kind of semi orthogonal matrix and its pseudo inverse is given by $A^{\dagger}=A^{T}$. Now you are trying to find a vector which is the orthogonal projection of $v$ in the space spanned by columns of $A$. This is also the closest vector to $v$ in the euclidean norm sense. Let us call it $v_{p}$. Clearly $v_{p}=Au$ for some vector $u$ in $\mathbb{R}^{m}$. So you are trying to solve \begin{align} \min_{u ~\epsilon~ \mathbb{R}^{m} } ||v-Au||^{2} \end{align} The solution for this is clearly $u=A^{\dagger}v=A^{T}v$. Substituting this for $v_p$, we obtain $v_p=AA^{T}v=Pv$.


Note that, we want to solve for a $u$ such that $v=Au$ (which is equivalent to that minimization problem). Now $A^{T}A=I$, since it is orthogonal. So multiplying by $A^{T}$ on both sides of $v=Au$ gives $u=A^{T}v$, and hence the required solution.

share|cite|improve this answer
Thank you very much, but I have not been taught yet of psuedo inverse. Is there another way to prove this without using that? – diimension Nov 3 '12 at 22:29
@diimension Please see the revised solution. – dineshdileep Nov 4 '12 at 4:16
Thank you very much, Dineshdileep! – diimension Nov 4 '12 at 5:26
Dineshdileep, sry to bother again on this post but I did not understand your edited proof. For ex. how where did $v=Au$, does $u$ correspond to $w$ come from Can you, if you don't mind, add more detail? – diimension Nov 17 '12 at 6:11
$w$ corresponds to $Au$ and not $u$. $w$ is the vector in the $span(A)$ (column span) which is closest to $v$ as possible. So for all $u \epsilon \mathbb{R}^{N}$, we find a $w=Au$ such that it is closest to $v$. Putting that equality constraint makes sure that $w$ lies in span(A) – dineshdileep Nov 18 '12 at 10:05

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.