2
$\begingroup$

geometrically, two vectors are "orthogonal" if there is a right angle between them, which we intuitively understand.

Algebraically, the definition of "orthogonal" members of a vector space, is that the dot product between the two vectors is zero. This means that for vectors $a,b$, it is the case that $\sum_{i=1}^na_i\cdot b_i=0$.

However, these coordinates depend on the chosen basis. If we take the standard Cartesian coordinate system, the bases are what we would intuitively call "orthogonal". However, the dot product is defined on the coordinates given a certain basis.

Does this mean that if we take a different basis, the vectors that are orthogonal will be different? For example, in the space of polynomials of order 2, if we take the bases $i = x, j = x^2$, then for example $(1,-1)\cdot (x,x^2)$ and $(1,1)\cdot (x, x^2)$ will be orthogonal, but if we take $i=x+x^2, j= x-x^2$, then those vectors will not be orthogonal.

Am I misunderstanding something here?

$\endgroup$
  • $\begingroup$ What is $(x, x^2)$? A pair of polynomials? If so, then $(1,-1)\cdot (x, x^2)$ is not what a dot product defines. A dot product defines the product between (in your case) two polynomials. $\endgroup$ – 5xum Mar 16 '17 at 12:33
  • $\begingroup$ I understand. That wasn't a dot product, that was just a set of coordinates multiplied by a set of basis vectors. $\endgroup$ – user56834 Mar 16 '17 at 12:42
  • $\begingroup$ Oh. Well, since the vectors in that case are polynomials, it would be better to just say $x-x^2$ and $x+x^2$... $\endgroup$ – 5xum Mar 16 '17 at 12:43
  • 2
    $\begingroup$ Orthogonality depends on a choice of dot product (also called scalar product), not on a choice of basis. Changing the basis should complicate a little bit how you calculate the usual dot product. $\endgroup$ – Olivier Mar 16 '17 at 12:55
  • 2
    $\begingroup$ @Olivier you should make this an answer, because the current answers have one thing in common: they are horrible, since they do not point out that orthogonality is intrinsically defined via a given map $V \times V \to k$ and thus does automatically NOT depend on the choice of a basis. You also lose a word about where the confusion comes from: The way to evaluate the map $V \times V \to k$ if one is given coordinates of the vectors, of course depends on the basis. $\endgroup$ – MooS Mar 16 '17 at 13:01
4
$\begingroup$

You are right, but it helps to look at the situation more abstractly.

First about vectors. Abstractly a vector is just an element of a vector space.

As you point out polynomials form a vector space and so it makes sense to say that $i = x + x^2$ is a vector. Similarly we can make sense of the claim perpetrated by physicists that anything with both a magnitude and a direction is a vector. However, when vectors are elements of an abstract vector space $V$, the dot-product (as given) does not make a whole lot of sense. Frankly, the only place where the dot product makes sense is in the very special vector space $\mathbb{R}^n$ whose elements are ordered tuples of numbers.

Now as you noticed there is a way out of this problem: by picking basis you get an isomorphism from any abstract vector space $V$ to some $\mathbb{R}^n$: just send any vector to its coordinate vector. And hence you can carry out your dot product after this isomorphism, giving you some sort of dot product (I come back to this later) on your abstract vector space $V$. And yes, as you point out, different choices of basis will generically give different products on your space - the other answers point out under which special circumstances different choices of basis will give the same product.

I wanted to say something else, namely how to retell this same story on an abstract vector space WITHOUT picking a basis.

The concept that plays the role of dot product is that of an inner product (or scalar product). Abstactly an inner product is a map $\langle . , . \rangle$ that eats two vectors and spits out one number, which is subject to 3 conditions:

  • Symmetry: $\langle \mathbf{v}, \mathbf{w} \rangle = \langle \mathbf{w}, \mathbf{v} \rangle$
  • Bilinearity: $\langle a \mathbf{u} + b \mathbf{v}, \mathbf{w} \rangle = a \langle \mathbf{u}, \mathbf{w} \rangle + b \langle \mathbf{v}, \mathbf{w} \rangle$
  • Possitive definiteness: $\langle \mathbf{v}, \mathbf{v} \rangle \geq 0$ for all vectors $\mathbb{v}$ in your vectorspace, with equality only if $\mathbf{v} = \mathbf{0}$.

Now the first thing you should verify is that when your abstract vector space happens to be $\mathbb{R}^m$, the ordinary dot product is in fact an example of an inner product in this abstract sense.

The much more interesting thing to verify is that this abstract notion matches our geometric understanding of dot products: that is if you define being orthogonal as having inner product equal to zero then the above three properties alone are enough to reproduce familiar properties of orthogonality, like the maximum number of mutually orthogonal vectors equals the dimension of the space.

Before returning to your question here is an interesting example: let $V$ be the vectorspace of all smooth functions on the closed interval $[0, 1]$. Now define an inner product by $$\langle f, g \rangle = \int_0^1 f(x)g(x) dx.$$ Then this map is indeed an inner product in the above definition! In particular we can use our geometric intuition about angles, distances etc to think about these functions before every having seen a basis for this space!

Now back to the question. Suppose you have an abstract vector space (or a concrete but non-geometric vector space such as the space of polynomials) and you want an inner product on your space in order to make use of your geometrical intuitions about orthogonality, angles, distances etc. There are really two ways to think about it:

1) First pick an inner product $\langle . , . \rangle$ on your space. This gives you everything you need. There are many choices, but some are more natural (such as the one defined by integration) and once you have chosen one once and for all it is determined what you mean by orthogonal. If later you want a basis, it would be nice to make it an orthonormal basis w.r.t. to this inner product.

2) First choose a basis. Then define an inner product by $\langle u, v \rangle$ = 'the dot product of the coordinate vectors of $u$ and $v$ w.r.t. the chosen basis'. This is indeed an inner product according to above abstract definition. You get for free that your basis is orthogormal. Again: different choices of basis lead to different inner products.

Which approach to take (1 or 2) depends on the context: sometimes a natural basis presents itself (as with $\mathbb{R}^n$) and sometimes it doesn't (such as with functions on $[0, 1]$).

However it is good to check how they are related: every abstract inner product (as in 1) can be found from a basis as in 2 and conversely for every basis there is an inner product making that basis orthonormal (that is what approach 2 says).

Also the set of inner products is smaller than the set of bases in the sense that many bases give rise to the same inner product: starting with the inner product (as in 1) makes it clear which are the bases that would give rise to that inner product (per construction 2): it are just all the bases that are orthonormal w.r.t. the given inner product.

$\endgroup$
  • 1
    $\begingroup$ How can you start your answer with the words "You are right", when the answer to the main question (the bold text) is clearly "No". $\endgroup$ – MooS Mar 16 '17 at 14:09
  • $\begingroup$ Well I mean it more in a 'you are right that something fishy is going on here' sort of way. I read the OP as saying that from a geometric perspective orthogonality should not depend on the choice of basis, while defining it in terms of dot products seems to lead to the conclusion that it does. Hence there is something that needs clarification. $\endgroup$ – Vincent Mar 16 '17 at 14:24
  • $\begingroup$ Is it also the case that every conceivable inner product is definable as the standard dot product of the n-tuple representation using some basis? $\endgroup$ – user56834 Mar 16 '17 at 16:03
  • $\begingroup$ Yes! I wrote that in the answer already, I believe? Ah yes. Second paragraph from the bottom. I say it a bit indirectly, but the answer is yes. You should try and write the proof yourself $\endgroup$ – Vincent Mar 17 '17 at 8:48
1
$\begingroup$
  • Orthogonality is a geometric property of two vectors.

  • Zero dot-product ($u.v=0$) is an algebraic property of the coordinates of two vectors relative to some basis.

Basically, the equivalence between orthogonality (right angles) and zero dot-product only holds for orthogonal bases.

A basis is orthogonal if, when you write its basic vectors in terms of the canonical (basic, standard, etc.) basis, these basic vectors are pairwise orthogonal (have zero dot-products).

$\endgroup$
1
$\begingroup$

Given a vector space equipped with a map $a:V \times V \to K$ (satisfying some conditions), $v$ and $w$ are called orthogonal if $a(v,w)=0$. This does of course NOT depend on the choice of a basis, since the definition does not need the presence of a basis.


The reason for your confusion is the following:

Given a basis $b_1, \dotsc, b_n$, one can define a matrix $M$ via $M_{ij} = a(b_i,b_j)$ and then one has that for $v=\sum v_ib_i, w=\sum w_ib_i$ one can compute $$a(v,w) =(v_1, \dotsc, v_n)M(w_1, \dotsc, w_n)^T$$, i.e. $M$ encodes the "formula" to compute $a(v,w)$ if one is given coordinates of $v,w$ with respect to the given basis. If you change the basis, of course $M$ changes. Your confusion arises because you changed the basis and then computed $a(v,w)$ with the matrix $M$ from the old basis. This of course gives you a wrong value for $a(v,w)$.

$\endgroup$
1
$\begingroup$

As has been pointed out by others, you don't need a basis to define orthogonality -- you just need an inner product, which is an abstract version of the "dot product".

However, having said that, it is true that if you have a basis, you can use that basis to define an inner product via the usual dot product -- and in that setting your question is a perfectly reasonable one.

So let me restate your question. Suppose you have a ($n$-dimensional) vector space $V$ with a basis $\mathcal{B}_1$. With respect to that basis, any vectors can be represented as $n$-tuples. Then you can define a dot product relative to that basis in the usual way, and relative to that dot product, some pairs of vectors will be orthogonal to one another.

Now change to a different basis, $\mathcal{B}_2$. With respect to this new basis, every vector has a different representation as an $n$-tuple. Relative to this second basis, we can define a different dot product, and relative to that new dot product, some pairs of vectors will be orthogonal to one another.

Your question is: Are the vectors that are orthogonal to one another relative to basis $\mathcal{B}_1$ also orthogonal to one another relative to basis $\mathcal{B}_2$?

(For those who insist that this is really a question about inner products, and that you don't need a basis to talk about an inner product, please note that the question is really about two different inner products on the same space, each one induced by a choice of basis. So the question can be reframed more abstractly as: when do two bases induce the same inner product?)

The answer is: In general, no. However, there is a specific set of conditions under which the answer is yes. That condition is:

If the vectors in $\mathcal{B}_2$ are orthonormal relative to the dot product define by $\mathcal{B}_1$, then orthogonality is preserved when changing basis.

Actually a more general property is true: if the vectors in $\mathcal{B}_2$ are orthonormal to one another relative to the dot product define by $\mathcal{B}_1$, then all dot products are preserved when changing basis.

Note that this condition is symmetric, because if the vectors in $\mathcal{B}_2$ are orthonormal to one another relative to the dot product define by $\mathcal{B}_1$, then also the vectors in $\mathcal{B}_1$ will be orthonormal to one another relative to the dot product define by $\mathcal{B}_2$.

Equivalently, you can express this condition in terms of the change of basis matrix $M$: We need the columns of $M$ to be an orthonormal set. This, in turn, can also be expressed as the condition $M^TM = I$. Matrices satisfying this condition are called orthogonal matrices. (A better terminology would probably be to call them orthonormal matrices, but unfortunately the terminology is standard.)

Here is a simple proof of why this condition is the one you need. Let $v,w$ be any two vectors in $V$. Relative to the basis $\mathcal{B}_1$, they can be represented as column vectors $\bf{v}_1$, $\bf{w}_1$, and the dot product is just given by $\bf{v}^T_1 \bf{w}_1$, where $T$ indicates matrix transposition and the product is just matrix multiplication. Similarly, relative to the basis $\mathcal{B}_2$, $v$ and $w$ can be represented as (different!) column vectors $\bf{v}_2$, $\bf{w}_2$, and the dot product is just given by $\bf{v}^T_2 \bf{w}_2$.

Now, the connection between these two representations is expressed by $\bf{v}_2 = \bf{M} \bf{v_1}$ and $\bf{w}_2 = \bf{M} \bf{w_1}$, where $\bf{M}$ is the change-of-basis matrix. Therefore we have: $$\begin{align} \bf{v}^T_2 \bf{w}_2 &= \left(\bf{Mv_1} \right)^T \bf{Mw_1} \\ &= \bf{v}^T_1 \bf{M}^T\bf{M} \bf{w}_1\\ &= \bf{v}^T_1 \left(\bf{M}^T\bf{M}\right) \bf{w}_1 \end{align}$$ so if the matrix $\bf{M}$ is orthogonal — that is, if $\bf{M}^T\bf{M}=\bf{I}$ — then we have $\bf{v}^T_2 \bf{w}_2 =\bf{v}^T_1 \bf{w}_1$, which shows that the dot product relative to the second basis produces the same value as the dot product relative to the first basis, and in particular that orthogonality is preserved when the basis is changed.

$\endgroup$
  • $\begingroup$ Genuinely curious why this was downvoted. $\endgroup$ – mweiss Mar 17 '17 at 13:57
  • $\begingroup$ I don't know. I upvoted so it's 0 now $\endgroup$ – user56834 Mar 17 '17 at 15:05
0
$\begingroup$

You have to apply base changes (from cartesian to another base) which preserve the angles locally. Not all base changes do that. A conformal map does.

$\endgroup$
0
$\begingroup$

For a linear transformation $$ x' = A x $$ then $$ x' \cdot y' = x'^T y' = x^T A^T A y =^! x^T y = x \cdot y $$ so you want $A^TA = I$, orthogonal matrices, as base transformation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.