# Creating Orthogonal, Orthonormal Vectors

I'm trying to understand gram-schmidt orthogonalization process. And it says $$p_1,p_2...p_N$$ are linearly independent vectors and $$q_1,q_2...q_N$$ are orthonormal vectors created from them. And also there are some equations like in the below:

$$q_1$$ is unit vector so it's length is 1 hence $$q_1 = p_1/ |p_1|$$ and also q1.q1 = 1 because it is unit. But there is an equation that I didn't understand. What means the $$q_2 = a*p_1 + b*p_2 = alpha*q_1 + beta*p_2$$ How was this equality achieved? Is a and b are coefficient/number or angle? What is the geometric explanation for this?

Can anybody explain this equation?

• I have edited my answer with more clarification. Commented May 29, 2021 at 18:06

Recall your statement :

And it says $$p_{1},p_{2},...,p_{N}$$ are linearly independent vectors and $$q_{1},q_{2},...,q_{N}$$ are orthonormal vectors $$\color{blue}{\text{created from them}}$$

Indeed, since this is the ultimate goal of the Gram-Schmidt process! To construct a set of orthonormal vectors from your original set of vectors as I highlighted your comment in blue. These two sets should share most of the vector properties.

 : To make things more clear, I will for the sake of simplicity reduce the set of LI vectors $$V$$ to just two vectors. Define : $$V:=\{v_{1},v_{2}\}$$ This set forms a basis in $$\mathbb{R}^{2}$$. We shall construct the desired set : $$Q:=\{q_{1},q_{2}\}$$ such that the elements of $$Q$$ are the orthonormal vectors of $$V$$ and note that : $$\operatorname{span}(V)=\operatorname{span}(Q)$$ The GSP begins by defining the first element of $$Q$$ that is $$q_{1}$$ : $$w_{1}:=v_{1}\implies q_{1}:=\frac{w_{1}}{\|w_{1}\|_{2}}=\frac{v_{1}}{\|v_{1}\|_{2}}$$ Next, let $$w_{2}:=v_{2}-\operatorname{proj}_{v_{1}}v_{2}$$. Geometrically, the projection is more of a "copy" of $$v_{2}$$ but by forcing it to be aligned in the same direction of $$v_{1}$$. It turns out that $$w_{2}$$ is formed by vector addition of $$v_{2}$$ and $$-\operatorname{proj}_{v_{1}}v_{2}$$. Therefore, what you get essentially is : \begin{align} w_{2}&:=v_{2}-\operatorname{proj}_{v_{1}}v_{2}\\ &=v_{2}-\frac{v_{1}^{\intercal}v_{2}}{v_{1}^{\intercal}v_{1}}v_{1}\\ &=v_{2}-\frac{v_{1}^{\intercal}v_{2}}{\|v_{1}\|_{2}^{2}}v_{1}\\ &=v_{2}-(q_{1}^{\intercal}v_{2})q_{1} \end{align} and what remains is normalizing the vector : $$q_{2}:=\frac{w_{2}}{\|w_{2}\|_{2}}=\frac{v_{2}-(q_{1}^{\intercal}v_{2})q_{1}}{\|v_{2}-(q_{1}^{\intercal}v_{2})q_{1}\|_{2}}$$ Now for what you are concerned about are the coefficients. Well note that : $$v_{1}=\|v_{1}\|_{2}q_{1}=c_{1}q_{1}$$ and \begin{align} v_{2}&=\|v_{2}-(q_{1}^{\intercal}v_{2})q_{1}\|_{2}q_{2}+(q_{1}^{\intercal}v_{2})q_{1}\\ &=c_{2}q_{2}+(q_{1}^{\intercal}v_{2})q_{1} \end{align} as you can see $$q_{1},v_{1}$$ and $$q_{2},v_{2}$$ are all related through $$c_{1}$$ and $$c_{2}$$ and this affirms our early statement that the span of $$V$$ and $$Q$$ must be the same. In fact, we can write : $$\mathbf{V}=\begin{bmatrix}v_{1}&v_{2}\end{bmatrix}_{2\times 2}=\begin{bmatrix}q_{1}&q_{2}\end{bmatrix}_{2\times 2}\begin{bmatrix}c_{1}&q_{1}^{\intercal}v_{2}\\ 0&c_{2}\end{bmatrix}_{2\times 2}=\mathbf{QR}$$ where $$Q^{\intercal}Q=I$$ (i.e., orthonormal matrix) and $$R$$ is an upper triangular $$\color{blue}{\text{invertible}}$$ matrix (it is clear why this matrix of coefficients must be invertible).

From this you can generalize this for all $$n$$ vectors of $$V$$ in $$\mathbb{R}^{n}$$.

• "second orthonormal vector is formed as a linear combination of the first two vectors of your original set of vectors" sentence is what I looking for. I thought of it as angular expressions that depend on their state in space. Commented May 24, 2021 at 17:57

I will elaborate what the author means. We can first normalize $$p_1$$ to get $$q_1 = \frac{1}{|p_1|} p_1$$ This $$\{ q_1 \}$$ is now a 1-element orthonormal set with the same span as $$p_1$$. It is trivially "orthogonal" and it is normal because $$\langle q_1 , q_1 \rangle = \langle \frac{1}{|p_1|} p_1, \frac{1}{|p_1|} p_1 \rangle = \frac{1}{|p_1|^2} \langle p_1, p_1 \rangle = \frac{|p_1|^2}{|p_1|^2} = 1$$ Now, let's move on to adding $$p_2$$ into our orthonormal collection. We want $$\{ q_1, q_2 \}$$ to have the same span as $$\{ p_1, p_2 \}$$, so if our desired $$q_2$$ exists, we need it to satisfy $$q_2 = a p_1 + bp_2 = \alpha q_1 + \beta p_2$$ for some $$a, b, \alpha, \beta \in \mathbb{R}$$ (or $$\mathbb{C}$$ if you are working with complex vector spaces). In order to ensure orthogonality we need $$0 = \langle q_1, q_2 \rangle = \langle q_1, \alpha q_1 + \beta p_2 \rangle = \alpha \langle q_1, q_1 \rangle + \beta \langle q_1, p_2 \rangle = \alpha + \beta \langle q_1, p_2 \rangle$$ In order to ensure that a possible $$q_2$$ is normal, we would need $$1 = \langle q_2, q_2 \rangle = \langle \alpha q_1 + \beta p_2, \alpha q_1 + \beta p_2 \rangle = \alpha^2+2 \alpha \beta \langle q_1, p_2 \rangle + \beta^2 |p_2|^2$$ Putting these conditions together, we need $$0 = \alpha + \beta \langle q_1, p_2 \rangle, 1=\alpha^2+2 \alpha \beta \langle q_1, p_2 \rangle + \beta^2 |p_2|^2$$ for the desired $$q_2$$ to exist. If you solve for $$\alpha, \beta$$, you will find that they do produce a valid $$q_2$$ (i.e. $$\{q_1, q_2 \}$$ is an orthonormal set with the same span as $$\{ p_1, p_2 \}$$), showing existence.

Edit: I am updating my answer with the new screenshot to follow their method. After seeing $$0 = \alpha + \beta \langle q_1, p_2 \rangle \implies \alpha = -\beta \langle q_1, p_2 \rangle$$ They substitute this into the desired expression for $$q_2$$: $$q_2 = \alpha q_1 + \beta p_2 = (-\beta \langle q_1, p_2 \rangle) q_1 + \beta p_2 = \beta (p_2 - \langle q_1, p_2 \rangle q_1)$$ They are now saying define vector $$r_2$$ to be $$p_2 - \langle q_1, p_2 \rangle q_1$$. Then, by our conditions and definitions, $$q_2$$ will be orthogonal to $$q_1$$ exactly when $$q_2 = \beta r_2$$ where $$\beta$$ is the $$\beta$$ we want to define $$q_2$$ with. Since we also want $$|q_2|=1$$, we also need $$1 = \langle q_2, q_2 \rangle = \langle \beta r_2, \beta r_2 \rangle = \beta^2 |r_2|^2 \implies \beta = \frac{1}{|r_2|} \implies q_2 = \frac{r_2}{|r_2|}$$ Unraveling our definitions, we have that $$q_2 = \frac{1}{|p_2 - \langle q_1, p_2 \rangle q_1|} (p_2 - \langle q_1, p_2 \rangle q_1)$$ and you can check that this $$\{ q_1, q_2 \}$$ indeed is an orthonormal set with the same span as $$\{ p_1, p_2 \}$$.

• This two answer helped me. I updated the image in the question. Can you look q2 = βr2 equation? I guess r2 other vector representation. Finally I want to ask that: q2.q2 = 1 so <βr2, βr2> = 1 how we get q2 = r2 / |r2| equation from it? Did we ignore β since it's a coefficient? And did we try to represent this in terms of r2? Commented May 24, 2021 at 17:22
• I have updated my answer to explain the further work in the new screenshot Commented May 24, 2021 at 17:39
• Thank you now I understand. And I didn't update my question but I want to ask that: there is expression like <q1,p3> = 0 and <q2, p3> = 0 why this way? Do I need image vectors on space or it's related to algebra and trigonometri? Commented May 24, 2021 at 19:30
• Not sure I understand what you mean. The algorithm continues very similarly for $p_3, p_4, ...$ etc. You can again just use the fact that you want $\langle q_3, q_2 \rangle = 0$, $\langle q_3, q_1 \rangle = 0$, and $\langle q_3, q_3 \rangle = 1$ to construct $q_3$ (that's just the definition of an orthonormal basis). It will work for arbitrarily high dimension Commented May 24, 2021 at 22:56
• I tried to explain my question without image but now I added it. If you look to question again you can see the point that I highlighted with blue. How q1.p3 and q2.p3 will be zero? What's the explanation of that? Commented May 25, 2021 at 10:39

If $$q_1 = \frac {p_1}{\vert p_1 \vert }$$ then $$aq_1 = a \frac {p_1}{\vert p_1 \vert } = \frac {a}{\vert p_1 \vert }p_1$$ so set $$\alpha = \frac{a}{\vert p_1 \vert }$$ and you're done.