I am confused in finding the projection of a vector on a subspace as I am new to it. I am able to find the orthogonal projection of a vector on another vector but I am not able to calculate orthogonal projection of a vector on to a subspace of vectors even though there is some literature that talks about it such as.

Projection of vector onto span

How to find orthogonal projection of vector on a subspace?

I have the following vectors:

$$ U= \begin{bmatrix} 2\\ 4\\ 8\\ \end{bmatrix} $$

$$ V= \begin{bmatrix} .4082 & .6667\\ .4082 & .6667\\ .8165 & .3333\\ \end{bmatrix} $$

1 I want to find the orthogonal projection of U on subspace V. What will be the orthogonal projection in this case and how is it found. Kindly explain?

2 What do you mean by subspace? Are these vectors in set V, in the same subspace or different and why?

3 Is there any difference between projection and orthogonal projection?

  • 1
    $\begingroup$ As you wrote, $\;V\;$ is a matrix, not a subspace, so: what do you mean? $\endgroup$ – DonAntonio Nov 28 '17 at 9:48
  • $\begingroup$ V is subspace. I am not clear with what subspace is, so asked about that. Thanks. $\endgroup$ – Navdeep Nov 28 '17 at 9:50
  • $\begingroup$ Some authors and teachers use "projection" as a synonym for "orthogonal projection". This is unfortunate, because there is a perfectly good notion of projection that includes (properly) orthogonal projection as a special case. Suppose $V$ is a direct sum of subspaces $K$ and $L$. Given $v \in V$ there are unique $v_{K} \in K$ and $v_{L} \in L$ such that $v = v_{K} + v_{L}$. The projection of $v \in V$ onto $L$ along $K$ is defined to be $v_{L}$. Given an inner product, the orthogonal projection onto $L$ is defined to be the projection onto $L$ along its orthogonal complement $L^{\perp}$. $\endgroup$ – Dan Fox Nov 28 '17 at 9:51
  • $\begingroup$ Ok, if you've decided it is subspace then continue alone as I, obviously, cannot help anyone not wishing to listen. $\endgroup$ – DonAntonio Nov 28 '17 at 9:51
  • $\begingroup$ $V$ is not a subspace. You might mean that $V$ is the space spanned by the two columns of that matrix. $\endgroup$ – Gerry Myerson Nov 28 '17 at 9:52

Given a unit column vector $u$ if I want to orthogonally project some other vector $v$ onto the subspace generated by $u$ I would first construct the matrix $P=uu^T$ then multiply $v$ on the left as $Pv$ and this will be the orthogonal projection of $v$ onto the subspace generated by $u$. This can be seen as follows. Firstly, to see that $Pv$ is in the span of $u$ we have $Pv=(uu^T)v = u(u^Tv)$ and since $u^Tv$ is the dot product of $u$ and $v$ it is a scalar and we have that $Pv$ is some scalar multiple of $u$, so it is in the span of $u$.

Now to see that this is the orthogonal projection we need to verify that the dot product of $Pv$ and $v-Pv$ is zero. To see this we will use the fact that $P^T=(uu^T)^T = u^T(u^T)^T = uu^T = P$, ie $P$ is symmetric and that $u$ is a unit vector so it's dot product $u^Tu=1$. Note now that $P^2= (uu^T)(uu^T)=u(u^Tu)u^T = uu^T = P$ So we now calculate $$(Pv)^T(v-Pv)=v^TP^T(v-Pv)=v^T(Pv-P^2v)=v^T(Pv-Pv) = 0$$ and we have the result.

From here all that is required is to adapt this technique to more than one unit vector, but this works particularly well because columns of a matrix act independently on a vector so rather than beginning with $u$ and calculating $uu^T$ we start with the matrix $U$ whose columns are an orthogonal basis for the subspace you wish to span and calculate $UU^T$. Note that the assumption that the column are orthogonal to each other is not optional and the Gram-Schmidt process can provide you such a basis.

A subspace $W$ of a vector space $V$ is a subset of $V$ that satisfies all the axioms of a vector space. Some examples would be any line or plane through the origin in $\mathbb{R}^3$. If I take any two vectors on one of these lines or planes and add them together I will stay on that line or plane. The same goes for scalar multiples of some vector in that subspace. Because all the scalar multiples and all the vector additions stay in that plane it's a subspace. It's common to say that a subspace is "closed" with respect to vector addition and scalar multiplication when we mean that those operations will keep vectors inside the subspace.

Orthogonal projections are distinct from a general projection. For a simple example consider the plane $\mathbb{R}^2$ and we will look at some projections onto the x-axis. The most obvious is the orthogonal projection given by $(x,y) \rightarrow (x,0)$ which zeroes out the y-axis. However I can choose any line through the origin and project onto the x-axis parallel to that line. These projections will not be orthogonal but still respect scalar multiplication and addition so they are a linear projections. Typically we want an orthogonal projection though, because it makes computation simpler.

  • $\begingroup$ Could you please explain with the help of examples. $\endgroup$ – Navdeep Nov 29 '17 at 4:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.