Let V be a finite dinner-product space, given a$ v_1,...v_n$ orthogonal bases, and $w_1,...w_n \in W $be subspace of V. Then $v_1=w_1$, $v_2= w_2-proj_w v_2$, $v_3= w_3-proj_w v_2- proj_w v_3$... I know that $proj_w v_i$ is the vector $v_i$ projected onto W subspace. How to intuitively understand Gram-Schmidt process, why as i gets bigger, we keep on subtracting more? I looked up the diagram on Gram-Schmidt, but still fails to get the sense.
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
When you project a vector onto a subspace (any dimension), and subtract the result, you get a vector orthogonal to the subspace. Projection onto an $n$-dimensional subspace, on the other hand, can be accomplished by adding the projections onto the $n$ individual members of an orthogonal basis.
Perhaps try an example. Project $(x,y,z)$ onto the $2$-dimensional subspace spanned by the unit vectors in the directions of the $x$ and $y$ axes. These two are often denoted $\bf{\vec i}$ and $\bf{\vec j}$. The projection in this case is of course $(x,y,0)$. And that's indeed $(x,y,z)-(x,0,0)-(0,y,0)=(x,y,z)-\operatorname{proj}_{\bf{\vec i}}(x,y,z)-\operatorname{proj}_{\bf{\vec j}}(x,y,z)$. After subtracting, and normalizing, you of course get $\bf{\vec k}=(0,0,1)$, to complete the standard basis.
$\begingroup$
$\endgroup$
2
What helped me was to just follow the algebra.
We claim that $\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$ is orthogonal to $\mathbf u$.
In which case $\mathbf {u}\cdot(\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u) = 0$
Try it out.
$\mathbf{u}\cdot \mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf{u}\cdot \mathbf{u} = \mathbf{u}\cdot \mathbf{v} - \mathbf{u}\cdot \mathbf{v} = 0$
-
$\begingroup$ then why it keeps adding on more projections? $\endgroup$– shineMar 10, 2020 at 4:52
-
1$\begingroup$ Next we need to figure out what to do to $\mathbf w$ so we subtract the projection of $\bf u$ onto $\bf w$ and we get a vector that is orthogonal to $\bf u$. Then we subtract the projection of the transformed vector $\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$ onto $\bf w$. We know that this vector is orthogonal to $\mathbf u$ so subtracting a multiple of it doesn't add anything in the $\bf u$ direction. And we know that it will be perpendicular to our transformed vector. $\endgroup$– Doug MMar 10, 2020 at 4:54
$\begingroup$
$\endgroup$
The article you linked is quite good and has a very nice animation
Say have two linearly independent vectors $\mathbf{u_1},\mathbf{u_2}$
You first calculate $proj_{\mathbf{u_1}}(\mathbf{u_2})$, the projection of $\mathbf{u_2}$ to the line generated by $\mathbf{u_1}$ which is calculated by $\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$
Now if we subtract it from $\mathbf{u_2}$ we'll get rid of the "common" part of the two vectors (that's where the Venn's diagram comes into play).
You can now easily prove that $\mathbf{v_1} = \mathbf{u_1},\space \mathbf{v_2} = \mathbf{u_2} -\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$ are orthogonal
How would you construct an inductive proof for $n$ vectors?
answered Mar 10, 2020 at 5:04