1
$\begingroup$

Let V be a finite dinner-product space, given a$ v_1,...v_n$ orthogonal bases, and $w_1,...w_n \in W $be subspace of V. Then $v_1=w_1$, $v_2= w_2-proj_w v_2$, $v_3= w_3-proj_w v_2- proj_w v_3$... I know that $proj_w v_i$ is the vector $v_i$ projected onto W subspace. How to intuitively understand Gram-Schmidt process, why as i gets bigger, we keep on subtracting more? I looked up the diagram on Gram-Schmidt, but still fails to get the sense.

Wiki for Gram-schmidt

$\endgroup$

3 Answers 3

4
$\begingroup$

When you project a vector onto a subspace (any dimension), and subtract the result, you get a vector orthogonal to the subspace. Projection onto an $n$-dimensional subspace, on the other hand, can be accomplished by adding the projections onto the $n$ individual members of an orthogonal basis.

Perhaps try an example. Project $(x,y,z)$ onto the $2$-dimensional subspace spanned by the unit vectors in the directions of the $x$ and $y$ axes. These two are often denoted $\bf{\vec i}$ and $\bf{\vec j}$. The projection in this case is of course $(x,y,0)$. And that's indeed $(x,y,z)-(x,0,0)-(0,y,0)=(x,y,z)-\operatorname{proj}_{\bf{\vec i}}(x,y,z)-\operatorname{proj}_{\bf{\vec j}}(x,y,z)$. After subtracting, and normalizing, you of course get $\bf{\vec k}=(0,0,1)$, to complete the standard basis.

$\endgroup$
2
$\begingroup$

What helped me was to just follow the algebra.

We claim that $\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$ is orthogonal to $\mathbf u$.

In which case $\mathbf {u}\cdot(\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u) = 0$

Try it out.

$\mathbf{u}\cdot \mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf{u}\cdot \mathbf{u} = \mathbf{u}\cdot \mathbf{v} - \mathbf{u}\cdot \mathbf{v} = 0$

$\endgroup$
2
  • $\begingroup$ then why it keeps adding on more projections? $\endgroup$
    – shine
    Commented Mar 10, 2020 at 4:52
  • 1
    $\begingroup$ Next we need to figure out what to do to $\mathbf w$ so we subtract the projection of $\bf u$ onto $\bf w$ and we get a vector that is orthogonal to $\bf u$. Then we subtract the projection of the transformed vector $\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$ onto $\bf w$. We know that this vector is orthogonal to $\mathbf u$ so subtracting a multiple of it doesn't add anything in the $\bf u$ direction. And we know that it will be perpendicular to our transformed vector. $\endgroup$
    – Doug M
    Commented Mar 10, 2020 at 4:54
1
$\begingroup$

The article you linked is quite good and has a very nice animation

Say have two linearly independent vectors $\mathbf{u_1},\mathbf{u_2}$

You first calculate $proj_{\mathbf{u_1}}(\mathbf{u_2})$, the projection of $\mathbf{u_2}$ to the line generated by $\mathbf{u_1}$ which is calculated by $\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$

Now if we subtract it from $\mathbf{u_2}$ we'll get rid of the "common" part of the two vectors (that's where the Venn's diagram comes into play).

Gram-Schmidt

You can now easily prove that $\mathbf{v_1} = \mathbf{u_1},\space \mathbf{v_2} = \mathbf{u_2} -\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$ are orthogonal

How would you construct an inductive proof for $n$ vectors?

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .