Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose that we have a vector $w = \begin{bmatrix} 3\\ 4.654\\ 3.34\\ 4.234\\ -1.23\\ \end{bmatrix}$

and we want to find an orthogonal to $w$. I have read about Gram-Schmidt process for me the most simple is the following one. Since the dot product should be $0$, i randomly fix 4 random elements and i compute their inner product $p$ with the first 4 dimensions of vector $w$. Then in order to find the last coefficient of the orthogonal vector i divide $p$ with -1.23 (the last coefficient). Is this a wrong method? And why Gram Schmidt is being preferred?

share|cite|improve this question
up vote 1 down vote accepted

The Gram-Schmidt process is a systematic way of finding a whole set of orthogonal vectors that form a basis for a space spanned by given vectors. In your case, you're given only one vector, and are tasked with finding another, and the procedure you mention would find two orthogonal vectors in a 5 dimensional space. How would you find a third orthogonal vector? Extending your logic, you would have a new vector with three fixed coordinates and two unknowns, then require it to be orthogonal to the first two, giving two equations for the two unknowns. You would solve the 2x2 system to find a third orthogonal vector. Then for a fourth, you would fix two coordinates and have three unknowns, and now solve a 3x3 system. Similar logic shows that the fifth orthogonal vector would require solving a 4x4 system of equations. What's worse, it's not trivial to pick coordinates to fix and randomly set their values to begin with; if its random, you might accidentally pick something already spanned by the basis you've built so far, in which case the procedure would break down.

The Gram-Schmidt process is simply a consistent way of doing the same thing, that is guaranteed to find an orthonormal basis. It is certainly simpler to think through and apply consistently when you are solving for an orthogonal basis. The only operations involved are dot products, vector sums, and scalar multiplication; you never have to solve a system of equations when you use it. These are the reasons that Gram-Schmidt is preferred.

share|cite|improve this answer
Is it possible to present an example in which the described solution would break down by randomly chosen coefficients that belong to the spanned basis? I can't get it very clearly – curious Jul 1 '13 at 15:37
The examples would seem contrived, and they are unlikely to occur in practice. The point is they CAN happen, which means the algorithm you describe is inherently unstable. Take $w=(1,0,0,0,0)^T$ and $w_2=(0,1,0,0,0)^T$ as a start. Now say you randomly pick a third vector of the form $w_3=(a,b,0,0,0)^T$. This meets your requirements, because I randomly fixed three coordinates and am looking for the values of two others. If try to solve $w\cdot w_3=0$ and $w_2\cdot w_3=0$ for the two constants $a$ and $b$, you'll see that there is no non-zero solution. – rajb245 Jul 1 '13 at 16:44
But i am a bit confused whether the Gram-Schmidt process is computing what i want. Correct me if am wrong: a set of vectors (x_1,x_2,x_3) each of 5 dimension. The Gram Schmidt process will find other 3 vectors y_1,y_2,y_3 that they would be in between orthogonal but not with the x1,x2,x3. Am i correct? In my case i want to find y1 that would be orthogonal to x1... – curious Jul 2 '13 at 13:41
I'm not sure I understand exactly what you mean in the following: "The Gram Schmidt process will find other 3 vectors y_1,y_2,y_3 that they would be in between orthogonal but not with the x1,x2,x3." What does it mean to be "in between orthogonal but not with"? Perhaps rephrasing would help. In the case you mention, $y_1,y_2,y_3$ would be three orthogonal unit vectors that span the same subspace as $x_1,x_2,x_3$. That is, any vector you could get by scaling and adding up the $x$ vectors, you could get by scaling and adding up the $y$ vectors. – rajb245 Jul 2 '13 at 13:50
It means that y1 would be orthogonal with y2 and with y3 but not with x1. Put it in another way: Would the dot product of <x y>=0 From what i understood the answer is no. So Gram Schmidt cannot find an orthogonal vector y given another x. IS that true? – curious Jul 2 '13 at 13:54

Your method should work unless $w$ happens to be perpendicular to the space of vectors with zero last coordinate. Then it will give you a trivial solution.

If you want more control over the solution, then you can find a parametrization of the solutions of the homogeneous system

$$ x\cdot w = 0$$

through row-reduction. This gives you generators of the space of solutions, and you can pick parameters to your liking.

share|cite|improve this answer
So when Gram-Schmidt should be used?Is there a limitation on my method? – curious Jul 1 '13 at 14:21
Gram Schmidt is the process of taking a set of vectors and finding an orthonormal basis. You simply want to find ONE vector orthogonal to your given vector. If you have any zeros in the $i^{th}$ coordinate, use $e_i$. Otherwise pick the vector $(-v_2, v_1, 0, 0, ...)$. – muzzlator Jul 1 '13 at 14:40

Here is how you advance. Assume the wanted vector is

$$ u = \begin{bmatrix} a\\ b\\ c\\ d\\ e\\ \end{bmatrix}, $$

then take the dot product $u.w=0$ and then solve the resulting system. See here.

share|cite|improve this answer
This is what i am doing as i described. I randomly choose a,b,c,d and then i take the dot product equal to $0$ to find e such that $u\cdot w =0 $ – curious Jul 1 '13 at 14:21
@curious: Yes, that's right. You will have one equation in 5 variables which results in infinite number of solutions. – Mhenni Benghorbal Jul 1 '13 at 14:22
The problem is when i do implement in python a script for doing so i evaluate the cosine of an angle between two vectors then i compute the cosine of their orthogonal as previously described but never the result is the same. I do have some errors – curious Jul 1 '13 at 14:24
@curious: Just take the dot product of two vectors. For example $(2,3).(g,h)=0\implies 2g+3h=0$. – Mhenni Benghorbal Jul 1 '13 at 14:30
What do you mean just take the dot product of two vectors?Instead of cosine to take the dot product? I think you don't understand my comment. – curious Jul 1 '13 at 14:32

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.