# Gram–Schmidt process with linear independent and dependent vector

Suppose that $u_1, u_2$ are linearly independent and that $u_3$ is in $\textrm{Span}\{u_1, u_2\}$. Suppose further that the Gram–Schmidt process is applied to $\{u_1, u_2, u_3\}$ to generate a new set $\{v_1, v_2, v_3\}$. What is $v_3$? Explain your answer.

So what I have already figured out it that since $u_3$ is in the span of $\{u1, u2\}$ and $u_3$ can be expressed as $u_3=c_1u_1+c_2u_2$.

So with the Gram–Schmidt process, we know $u_1=v_1$. Then $v_2=u_2-\frac{u_2*v_1}{v_1*v_1}*v_1$

And then $v_3=u_3-\frac{u_3*v_1}{v_1*v_1}*v_1$

But I can express $u_3$ as $c_1u_1+c_2u_2$ so I can replace it in the $v_3$. And I can replace all the $v_1$ with $u_1$.

Therefore $v_3=c_1u_1+c_2u_2-\frac{c_1u_1+c_2u_2*u_1}{u_1*u_1}*u_1$

Does this mean that $v_3$ is also in the span of $\{u_1,u_2\}$ since it is expressed in terms of $u_1$ and $u_2$?

• Thanks for the editing Arthur. Dec 5 '13 at 4:51
• You're welcome. For future reference, here is a guide to typing math equations. Dec 5 '13 at 4:57

$v_3=u_3 - [(u_3|v_1)/(||v_1||^2)] \times v_1 - [(u_3|v_2)/(||v_2||^2)] \times v_2$

This is the Gram-Schmidt Orthogonalisation equation. If you apply this you would get answer straight away.

Since $u_3 \in \langle u_1,u_2\rangle$, Therefore, $u_3 \in \langle v_1,v_2\rangle$, because both $v_1=u_1 \; and \; v_2$ is a linear combination of $u_1,u_2$. Therefore $u_3 = a.v_1 + b.v_2$

Now $v_3 = a.v_1 + b.v_2 - (((a.v_1 + b.v_2)| v_1/||v_1||^2)\times v_1) + (((a.v_1 + b.v_2)| v_2/||v_2||^2)\times v_2) \\ = (a.v_1 + b.v_2) - (a.v_1 + b.v_2) = 0$

Remember that $(v_1|v_2) = 0$

$v_3$ will be zero (in infinite precision arithmetic).

This is because, by construction, $v_3 \in ( \operatorname{sp} \{ v_1, v_2 \} )^\bot$.

Note: The Gram Schmidt process takes a list of vectors $v_1, v_2,...$ and constructs a new list $\hat{u_1}, \hat{u_2}, ...$ as follows:

$\hat{u_1} = \frac{1}{\|v_1\|} v_1$, so $\operatorname{sp} \{ v_1 \} = \operatorname{sp} \{ \hat{u_1} \}$.

Now suppose we have $\hat{u_1},..., \hat{u_k}$ orthonormal such that $\operatorname{sp} \{ v_1,..,v_k \} = \operatorname{sp} \{ \hat{u_1},..., \hat{u_k} \}$. Then let $u_{k+1} = v_{k+1} -\sum_{i=1}^k \langle \hat{u_i}, v_{k+1} \rangle \hat{u_i}$, and $\hat{u}_{k+1} = \frac{1}{\|u_{k+1}\|} u_{k+1}$ (assuming that $u_{k+1}$ is non-zero).

It should be clear that we always have $u_{k+1} \in ( \operatorname{sp} \{ \hat{u_1},..., \hat{u_k} \} )^\bot$.

If it happens that $v_{k+1} \in \operatorname{sp} \{ \hat{u_1},..., \hat{u_k} \}$, then we must (in infinite precision arithmetic) have $u_{k+1} = 0$.

• So I'm trying to understand why v3∈(sp{v1,v2})⊥. Is it because v1 and v2 are linearly independent due to the original linearly independent u1 and u2 vectors? Dec 5 '13 at 5:07
• Well, the whole point of the GS process is to take a set of linearly independent vectors and orthogonalize them. The first vector is just normalized. The $k+1$th vector is constructed by subtracting off the projections onto the subspace spanned by the $1,...,k$th vectors. Hence the remainder will be orthogonal to the previous vectors. If it happens that a vector is in the span of the previous ones, then the resultant will be zero (ideally, but unlikely numerically). Dec 5 '13 at 5:29
• I added some detail to the answer. Dec 5 '13 at 5:49