0
$\begingroup$

I got a subspace $W=[(1,-3,0,1),(1,5,2,3),(0,4,1,1),(1,-2,0,4)]$

and I want to make an orthogonal projection of a vector $p=(1,0,0,0)$ onto $W$ and onto the orhhogonal complement of $W$.

Well, $W$ is $[w_1,w_2,w_3]=[(1,-3,0,1),(0,4,1,1),(1,-2,0,4)]$ for example and the orthogonal complement of $W$ is $W' = [(-10,-3,11,1)]$

So, we can project $p$ onto $W$ by the formula: $$p_{\| W} = \sum_{i =1}^k \frac{\langle p, w_i\rangle}{\langle w_i, w_i \rangle} w_i = \frac{\langle p , w_1 \rangle}{\langle w_1 , w_1 \rangle} w_1 + \dots + \frac{\langle p , w_k \rangle}{\langle w_k , w_k \rangle} w_k.$$

But I am somehow confused. Do the vectors $w_i$ have to be orthogonal to each other before I do the projection with the vector $p$? Because $W$ is not orthogonal itself. And examples I found contained the orthogonal subspaces already (so no Gramm-Schmidt process was needed).

$\endgroup$
  • 1
    $\begingroup$ Yes, the basis of $W$ must be orthogonalized first. You could alternatively find the projection of $p$ onto $W_\perp$ and subtract that from $p$ to get its projection onto $W$. This should be faster since $W_\perp$ is unidimensional. $\endgroup$ – Shubham Johri May 17 at 14:55
  • $\begingroup$ Thanks. And do I make the projection onto $W$ and $W'$ separately and then sum them? Or no summation? So the result would be two projected vectors? $\endgroup$ – Leif May 17 at 15:06
  • $\begingroup$ I saw a similar example with the direct sum of the subspaces so this confuses me too. $\endgroup$ – Leif May 17 at 15:07
  • $\begingroup$ $$p=p_{||W}+p_{||W_\perp}$$where $p_{||W_\perp}=\dfrac{\langle p, w\rangle}{\langle w, w\rangle} w$ and $w$ is the basis vector of $W_\perp$. $\endgroup$ – Shubham Johri May 17 at 15:46
  • $\begingroup$ Thank you for making it clearer for me! $\endgroup$ – Leif May 17 at 16:07
1
$\begingroup$

It is possible to project without an orthonormal basis.

Our space of interest $W$ is the column space of the matrix $A$ given by $$ A=\left[\begin{array}{rrrr} 1 & 1 & 0 & 1 \\ -3 & 5 & 4 & -2 \\ 0 & 2 & 1 & 0 \\ 1 & 3 & 1 & 4 \end{array}\right] $$ A basis of $W=\operatorname{Col}(A)$ is given by the columns of $A$ corresponding to pivot columns in $\operatorname{rref}(A)$. The reduced row echelon form of $A$ is $$ \operatorname{rref}(A)=\left[\begin{array}{rrrr} 1 & 0 & -\frac{1}{2} & 0 \\ 0 & 1 & \frac{1}{2} & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{array}\right] $$ This shows that $A$ has rank three and that the vectors \begin{align*} \left\langle1,\,-3,\,0,\,1\right\rangle && \left\langle1,\,5,\,2,\,3\right\rangle && \left\langle1,\,-2,\,0,\,4\right\rangle \end{align*} form a basis of $W$.

Now, put these basis vectors into the columns of a matrix $$ X = \left[\begin{array}{rrr} 1 & 1 & 1 \\ -3 & 5 & -2 \\ 0 & 2 & 0 \\ 1 & 3 & 4 \end{array}\right] $$ The projection matrix onto $W$ is $$ P=X(X^\top X)^{-1}X^\top $$ In our case, we end up with $$ P=\left[\begin{array}{rrrr} \frac{131}{231} & -\frac{10}{77} & \frac{10}{21} & \frac{10}{231} \\ -\frac{10}{77} & \frac{74}{77} & \frac{1}{7} & \frac{1}{77} \\ \frac{10}{21} & \frac{1}{7} & \frac{10}{21} & -\frac{1}{21} \\ \frac{10}{231} & \frac{1}{77} & -\frac{1}{21} & \frac{230}{231} \end{array}\right] $$ We can now project any vector $v\in\Bbb R^4$ onto $W$ by computing $Pv$.

If you're dead set on using orthonormal bases, then you could take these three basis vectors and apply the Gram-Schmidt algorithm. This gives the new basis \begin{align*} q_1 &= \left\langle\frac{1}{\sqrt{11}},\,-\frac{3}{\sqrt{11}},\,0,\,\frac{1}{\sqrt{11}}\right\rangle & q_2 &= \left\langle\frac{1}{\sqrt{7}},\,\frac{1}{\sqrt{7}},\,\frac{1}{\sqrt{7}},\,\frac{2}{\sqrt{7}}\right\rangle & q_3 &= \left\langle-\frac{1}{\sqrt{3}},\,0,\,-\frac{1}{\sqrt{3}},\,\frac{1}{\sqrt{3}}\right\rangle \end{align*} We can then project any $v\in\Bbb R^4$ with the formula $$ Pv=\langle q_1, v\rangle q_1 + \langle q_2, v\rangle q_2 + \langle q_3, v\rangle q_3 $$

$\endgroup$
  • $\begingroup$ A then I do the projection on W' and sum both projections, right? As was suggested above in the comments. $\endgroup$ – Leif May 17 at 16:12
  • 1
    $\begingroup$ @Leif The sum of the projections of $p$ onto $W$ and $W_\perp$ will just be $p$, as I mentioned. Why do you want to add them? The question only asks for the projections. $\endgroup$ – Shubham Johri May 17 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.