# Graphical explanation for orthogonalization

Edit: Perhaps a little too much background info, for the actual question please scroll down.

I was thinking how to come up with a graphical explanation for the Gram-Schmidt orthogonalization (as it would be more intuitive for people new to the method). However, I ran intro a problem in the case where your vectors span the $\mathbb{R}^3$.

As for the $\mathbb{R}^2$ it's easy:

In order to get $v_2$ from the known vector $v$ and $w_1$, observe that $v = v_1 + v_2$ and hence $v_2 = v - v_1$. Therefore we have to find $v_1$.

Observe that the length of $v_1$ , i.e. $\|v_1\|$, equals $\|v\| \; cos \varphi$. The direction of $v_1$ is the same as the first orthogonal vector $w_1$. So $v_1 = (\|v\| \; cos \varphi) \dfrac{w_1}{\|w_1\|}$.

This looks a little like an inner product, let's write it like one.

\begin{align} v_1 = (\|v\| \; cos \varphi) \dfrac{w_1}{\|w_1\|} = (\|v\| \; cos \varphi) \dfrac{w_1}{\|w_1\|} \dfrac{\|w_1\|}{\|w_1\|} = \dfrac{<v,w_1>}{<w_1,w_1>}w_1 \end{align}

And therefore, $v_2 = v - v_1 = v - \dfrac{<v,w_1>}{<w_1,w_1>}w_1$.

Now for $\mathbb{R}^3$, the same idea.

Because $u_3 = v_3 - y$, we have to find $y$. Well, $y = y_1 + y_2$. The length of $y$ equals $\|v_3\| cos \theta$.

Therefore, $y_1 = \|v_3\| cos \theta \; cos \varphi \dfrac{w_1}{\|w_1\|}$, and $y_2 = \|v_3\| cos \theta \; sin \varphi \dfrac{w_2}{\|w_2\|}$.

But from the normal derivation of Gram-Schmidt, $y_1$ should be $\dfrac{<v_3,w_1>}{<w_1,w_1>}w_1$ and $y_2$ should be $\dfrac{<v_3,w_2>}{<w_2,w_2>}w_2$.

So my question, how can I get to this result? Say the angle between $v_3$ and $w_1$ is $\psi$, then from the above equations for $y_1$ it follows that $cos\psi = cos\varphi \; cos \theta$...?

Edit: So the actual question is, why is $cos\psi = cos\varphi \; cos \theta$? Theta $(\theta)$ is the angle between $v_3$ and $y$, phi $(\varphi)$ is the angle between $y$ and $w_1$. Psi $(\psi)$ would be the angle between $v_3$ and $w_1$.

Now I think of it, I could've saved myself and you as reader some time since the actual problem only has to do with angles. Like I said, I know two angles $\varphi$ and $\theta$, how can I get $\psi$ from them such that the two expressions for $y_1$ are the same (and a similar question for $y_2$)?

-
You always subtract the projection of your vector on the other vectors and then normalize. This is a simple geometric interpretation... – Listing Nov 1 '11 at 23:12
@Listing: I agree, but in my experience students sometimes need some examples before they grasp the general idea (I'm talking about first year Mechanical Engineering students). – Ailurus Nov 1 '11 at 23:17
@Ailurus: An explanation that would be more intuitive than what, exactly? The interpretation that Listing mentioned is the standard one, and it's as geometric and intuitive as it can possibly be. (Does anyone ever try to explain Gram–Schmidt without mentioning that interpretation?) – Hans Lundmark Nov 2 '11 at 7:00
@Hans: Well, I used to do some tutoring for linear algebra, and the first year students are not that familiar with "Projection of ... on ...", or splitting inner products for that matter. Therefore I would like to use some geometry to show the results from their book which uses the standard explanation. Anyway, I did not come here for a nice discussion, but for a possible answer of my question. Thanks. – Ailurus Nov 2 '11 at 7:40

The inner product $<v_3,y>$ can be written as $<v_3,y_1+y_2> = <v_3,y_1> + <v_3,y_2>.$ Furthermore $<v_3,y> = \|v_3\|\|y\| \cos\theta$, $<v_3,y_1> = \|v_3\|\|y_1\| \cos \psi = \|v_3\|\|y\| \cos\varphi \cos \psi$ and $<v_3,y_2> = \|v_3\|\|y_2\| \cos\omega = \|v_3\|\|y\| \sin\varphi \cos \omega$.
Now, divide by $\|v_3\|\|y\|$ such that $\cos \theta = \cos \varphi \cos \psi + \sin \varphi \cos \omega$. Write $\cos \theta$ as $\cos \theta (\cos^2\varphi + \sin^2 \varphi) = \cos \theta \cos^2\varphi + \cos\theta \sin^2\varphi$.
Therefore, $\cos \psi = \cos \theta \cos \varphi$ (and $\cos \omega = \cos \theta \sin \varphi$).