# Intuition for Gram-Schmidt process

Let V be a finite dinner-product space, given a$$v_1,...v_n$$ orthogonal bases, and $$w_1,...w_n \in W$$be subspace of V. Then $$v_1=w_1$$, $$v_2= w_2-proj_w v_2$$, $$v_3= w_3-proj_w v_2- proj_w v_3$$... I know that $$proj_w v_i$$ is the vector $$v_i$$ projected onto W subspace. How to intuitively understand Gram-Schmidt process, why as i gets bigger, we keep on subtracting more? I looked up the diagram on Gram-Schmidt, but still fails to get the sense.

Wiki for Gram-schmidt

When you project a vector onto a subspace (any dimension), and subtract the result, you get a vector orthogonal to the subspace. Projection onto an $$n$$-dimensional subspace, on the other hand, can be accomplished by adding the projections onto the $$n$$ individual members of an orthogonal basis.

Perhaps try an example. Project $$(x,y,z)$$ onto the $$2$$-dimensional subspace spanned by the unit vectors in the directions of the $$x$$ and $$y$$ axes. These two are often denoted $$\bf{\vec i}$$ and $$\bf{\vec j}$$. The projection in this case is of course $$(x,y,0)$$. And that's indeed $$(x,y,z)-(x,0,0)-(0,y,0)=(x,y,z)-\operatorname{proj}_{\bf{\vec i}}(x,y,z)-\operatorname{proj}_{\bf{\vec j}}(x,y,z)$$. After subtracting, and normalizing, you of course get $$\bf{\vec k}=(0,0,1)$$, to complete the standard basis.

What helped me was to just follow the algebra.

We claim that $$\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$$ is orthogonal to $$\mathbf u$$.

In which case $$\mathbf {u}\cdot(\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u) = 0$$

Try it out.

$$\mathbf{u}\cdot \mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf{u}\cdot \mathbf{u} = \mathbf{u}\cdot \mathbf{v} - \mathbf{u}\cdot \mathbf{v} = 0$$

• then why it keeps adding on more projections? Commented Mar 10, 2020 at 4:52
• Next we need to figure out what to do to $\mathbf w$ so we subtract the projection of $\bf u$ onto $\bf w$ and we get a vector that is orthogonal to $\bf u$. Then we subtract the projection of the transformed vector $\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$ onto $\bf w$. We know that this vector is orthogonal to $\mathbf u$ so subtracting a multiple of it doesn't add anything in the $\bf u$ direction. And we know that it will be perpendicular to our transformed vector. Commented Mar 10, 2020 at 4:54

The article you linked is quite good and has a very nice animation

Say have two linearly independent vectors $$\mathbf{u_1},\mathbf{u_2}$$

You first calculate $$proj_{\mathbf{u_1}}(\mathbf{u_2})$$, the projection of $$\mathbf{u_2}$$ to the line generated by $$\mathbf{u_1}$$ which is calculated by $$\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$$

Now if we subtract it from $$\mathbf{u_2}$$ we'll get rid of the "common" part of the two vectors (that's where the Venn's diagram comes into play).

You can now easily prove that $$\mathbf{v_1} = \mathbf{u_1},\space \mathbf{v_2} = \mathbf{u_2} -\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$$ are orthogonal

How would you construct an inductive proof for $$n$$ vectors?