Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am trying to understand how - exactly - I go about projecting a vector onto a subspace.

Now, I know enough about linear algebra to know about projections, dot products, spans, etc etc, so I am not sure if I am reading too much into this, or if this is something that I have missed.

For a class I am taking, the proff is saying that we take a vector, and 'simply project it onto a subspace', (where that subspace is formed from a set of orthogonal basis vectors).

Now, I know that a subspace is really, at the end of the day, just a set of vectors. (That satisfy properties here). I get that part - that its this set of vectors. So, how do I "project a vector on this subspace"?

Am I projecting my one vector, (lets call it a[n]) onto ALL the vectors in this subspace? (What if there is an infinite number of them?)

For further context, the proff was saying that lets say we found a set of basis vectors for a signal, (lets call them b[n] and c[n]) then we would project a[n] onto its signal subspace. We project a[n] onto the signal-subspace formed by b[n] and c[n]. Well, how is this done exactly?..

Thanks in advance, let me know if I can clarify anything!

P.S. I appreciate your help, and I would really like for the clarification to this problem to be somewhat 'concrete' - for example, something that I can show for myself over MATLAB. Analogues using 2-D or 3-D space so that I can visualize what is going on would be very much appreciated as well.

Thanks again.

share|improve this question
    
Hopefully, this can help clear some of your confusion: A projection of a space $V$ onto a subspace $W$ is a map, $\pi:V \to W$, such that $\pi^2=\pi$. So given any vector $v\in V$, we have $\pi(v)\in W$ and $\pi(w)=w$ if $w\in W$. So a projection is a way of associating a vector in a subspace with each vector in the whole space in such a way that vectors in the subspace are associated with themselves. –  Bill Cook Feb 24 '12 at 3:04
    
@BillCook Thank you for your comment, unfortunately I do not understand how this helps me project onto a subspace. In one hand I have a vector a[n]. In the other hand I have two basis vectors of the subspace, b[n], and c[n]. How do I project a[n] on the subspace formed by those other two? –  Mohammad Feb 24 '12 at 3:11
add comment

2 Answers 2

up vote 5 down vote accepted

I will talk about orthogonal projection here.

When one projects a vector, say $v$, onto a subspace, you find the vector in the subspace which is "closest" to $v$. The simplest case is of course if $v$ is already in $v$, then the projection of $v$ onto the subspace is $v$ itself.

Now, the simplest kind of subspace is a one dimensional subspace, say the subspace is $U = \operatorname{span}(u)$. Given an arbitrary vector $v$ not in $U$, we can project it onto $U$ by $$v_{\| U} = \frac{\langle v , u \rangle}{\langle u , u \rangle} u$$ which will be a vector in $U$. There will be more vectors than $v$ that have the same projection onto $U$.

Now, let's assume $U = \operatorname{span}(u_1, u_2, \dots, u_k)$ and, since you said so in your question, assume that the $u_i$ are orthogonal. For a vector $v$, you can project $v$ onto $U$ by $$v_{\| U} = \sum_{i =1}^k \frac{\langle v, u_i\rangle}{\langle u_i, u_i \rangle} u_i = \frac{\langle v , u_1 \rangle}{\langle u_1 , u_1 \rangle} u_1 + \dots + \frac{\langle v , u_k \rangle}{\langle u_k , u_k \rangle} u_k.$$

Now, I don't know if this helps you (does your vector space come equipped with an inner product $\langle \cdot , \cdot \rangle$?).

share|improve this answer
    
Ah! Yes thanks this is very nice. So simply the new projected vector $p[n] = a[n]^{T}b[n] + a[n]^{T}c[n]$ will be the component of a[n] on the hyper-plane that is the subspace that b[n] and c[n] make together. Is this correct? –  Mohammad Feb 24 '12 at 3:23
    
I don't know if your formula for $p[n]$ is correct, but you geometric interpretation of is correct. –  Calle Feb 24 '12 at 3:26
    
The way I currently understand it, I take dot product of a[n] with b[n] and c[n], and I get two scores. Then I take the respective scores and go back and use them as weights on b[n] and c[n], and I add those now weighted basis vectors together. That is the projection of a[n] onto the hyperplane... –  Mohammad Feb 24 '12 at 3:30
    
Yes, that is correct. –  Calle Feb 24 '12 at 3:31
add comment

Take a basis $\{v_1, \dots, v_n\}$ for the "signal subspace" $V$. Let's assume $V$ is finite dimensional for simplicity and practical purposes, but you can generalize to infinite dimensions. Let's also assume the basis is orthonormal.

The projection of your signal $f$ onto the subspace $V$ is just

$$\mathrm{proj}_V(f) = \sum_{i=1}^n \langle f, v_i \rangle v_i$$

and $f = \mathrm{proj}_V(f) + R(f)$, where $R(f)$ is the remainder, or orthogonal complement, which will be 0 if $f$ lies in the subspace $V$.

The $i$-th term of the sum, $\langle f, v_i\rangle$, is the projection of $f$ onto the subspace spanned by the $i$-th basis vector. (Note, if the $v_i$ are orthogonal, but not necessarily orthonormal, you must divide the $i$-th term by $\|v_i\|^2$.)

share|improve this answer
    
Thank you kind sir, that was informative. You said: "The i-th term of the sum, ⟨f,vi⟩, is the projection of f onto the subspace spanned by the i-th basis vector." - and the subspace here is just the set made up of all scalar multiples of the $i-th$ basis, yes? –  Mohammad Feb 24 '12 at 3:39
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.