2
$\begingroup$

Let's say that we have a set of $n$ vectors, each belonging to $R^m$ that do not form a basis for $R^m$.

Intuitively, I feel like the space spanned by a set of orthogonal vectors will be greater than the space spanned by the same amount of linearly independent vectors. Now although this might now always be true, it intuitively feels like for the majority of cases this is true, as the volume of orthogonal vectors will be larger than the volume of linearly independent vectors (assuming magnitude if vectors are the same).

Can anyone shine some light on this intuition?

$\endgroup$
  • 1
    $\begingroup$ What does "greater" mean? $\endgroup$ – vadim123 Nov 29 '16 at 4:46
  • $\begingroup$ @vadim123 The cardinality of the set of vectors that can be made via a linear combination of the respective set. $\endgroup$ – Armen Aghajanyan Nov 29 '16 at 4:47
  • 1
    $\begingroup$ @ArmenAghajanyan $|\mathbb R^n|=|\mathbb R^m|$, so cardinality isn’t a good choice for defining “greater.” The dimension of the spanned space might be a better choice. $\endgroup$ – amd Nov 29 '16 at 8:44
4
$\begingroup$

Not true. Suppose you are in the regular 3D space $(\Bbb R^3)$. Two linearly independent vectors will span a plane, independent if those two vectors are orthogonal. In fact, if you have a set of linearly independent vectors, you can use them to construct the same number of orthogonal vectors

$\endgroup$
  • $\begingroup$ Makes sense. Does the same argument apply when the set of vectors do not form a basis. So the cardinality of the set of vectors is much less than the dimensionality of the vectors. And is the process you are referring to the Gram-Schmidt process? $\endgroup$ – Armen Aghajanyan Nov 29 '16 at 4:49
  • $\begingroup$ Yes. The fewer independent vectors will form a basis for the subspace spanned by them. And yes, I was referring to the Gram–Schmidt method $\endgroup$ – Andrei Nov 29 '16 at 4:53
  • $\begingroup$ This is a good example, but the point is general. $n$ linearly independent vectors will span an $n$ dimensional subspace. You can find a set of $n$ orthogonal vectors that span the same subspace, and in any case $n$ orthogonal vectors will span an $n$ dimensional subspace (unless one is zero-some definitions of orthogonal permit that). It seems reasonable to accept that all $n$ dimensional subspaces are the same size in the sense we are talking. $\endgroup$ – Ross Millikan Nov 29 '16 at 5:10
  • $\begingroup$ @ArmenAghajanyan The cardinality of all non-trivial subspaces of $\mathbb R^3$ is the same. Perhaps you should compare dimensions instead. $\endgroup$ – amd Nov 29 '16 at 8:45
2
$\begingroup$

In $\mathbb{R}^m$, no, the orthogonality doesn't matter.

However, hold onto this idea for later. In infinite dimensions you can construct a sequence of vectors that get closer and closer to each other, while still being linearly independent, and then there are some technical senses in which you can say they span a smaller space than an orthonormal set.

Also, in numerical linear algebra the "conditioning" of a set of vectors (a measure of how far they are from being orthonormal) is a really important concept. When a set of vectors is very ill-conditioned, although in principle it spans a given space, in practice it turns out that you can only effectively operate on a smaller space due to the amplification of numerical rounding errors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.