15
$\begingroup$

If you have two vectors living in subspace $V$ and you want to take dot product, it seems that you cannot technically do this operation because if you write both vectors in matrix form, they would both be column vectors living in the same subspace. In order to take the dot product, you would need to convert one of the vectors into a row vector which lives in a completely different dual subspace $V^*$ and then take the dot product of this dual space vector with the column vector. Is all of this true?

$\endgroup$
  • 3
    $\begingroup$ Are you sure you're talking about dot product and not matrix multiplication? $\endgroup$ – norvia May 12 at 0:06
  • 1
    $\begingroup$ At one level this might be a confusion over notation. But there's also an interesting general relationship between vector dual spaces and bilinear inner products! $\endgroup$ – aschepler May 12 at 0:21
  • $\begingroup$ "[...] you cannot technically do this operation because if you write both vectors in matrix form [...]": the usual inner product does not require you to rewrite the vectors in matrix form, and if you do so, what prevents you from accepting the definition $(a_1,...,a_n)\cdot(b_1,...,b_n)=a_1b_1+\cdots + a_n b_n$? $\endgroup$ – M. Rumpy May 12 at 19:15
  • $\begingroup$ @norvia: the point is "consistency with matrix-multiplication rules". If you transpose the vector on the left you end up with a $ 1 \times 1$ matrix (a number). If you transpose the vector on the right you end up with a $n \times n $ matrix... but this is not the dot product :) $\endgroup$ – mrc ntn May 27 at 16:00
19
$\begingroup$

You're right that there's something going on here.

In a general finite-dimensional vector space, there is no canonical choice of isomorphism from $V$ to $V^*$, even though they're isomorphic because they have the same dimension. However, in a general finite-dimensional vector space, there is also no canonical choice of inner product!

Having an inner product gives us an isomorphism $\phi: V \to V^*$: map a vector $v \in V$ to the element $w \mapsto \langle v,w\rangle$ in $V^*$, and we can check that this will be an isomorphism.

Going the other way is a bit trickier, since inner products need to satisfy $\langle v,v \rangle \ge 0$, but isomorphisms "don't know" about this structure. (In particular, for vector spaces over finite fields, we can have an isomorphism $\phi : V \to V^*$, but it doesn't make sense to have an inner product.) However, if you have an isomorphism $\phi : V \to V^*$, then you can define $\langle v, w\rangle = \phi(v)(w)$, and this will at least be a bilinear form. (Making it artificially symmetric is easy and left as an exercise.)

When we're talking about vectors written as column vectors, we've actually given our vector space lots of structure: we've picked a standard basis, and we're writing our vectors in terms of their coordinates in that basis. Here, taking the transpose to turn a column vector into a row vector is exactly the isomorphism that corresponds to taking the dot product as our inner product.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ If we're pedantic, we write something like $\langle v,\,w\rangle=(\phi(v)w)_{11}$. $\endgroup$ – J.G. May 12 at 14:05
  • $\begingroup$ @J.G. Oh, I was thinking that in this case, $V^*$ is the set of maps $V \to \mathbb R$, and so $\langle v, w\rangle = \phi(v)(w)$ is the function $\phi(v)$ evaluated at $w$. $\endgroup$ – Misha Lavrov May 12 at 16:06
  • 1
    $\begingroup$ Even better! But I'll leave my comment in place, in case other people ever misunderstand $v^Tw$ as being a scalar, rather than a $1\times1$ matrix. $\endgroup$ – J.G. May 12 at 16:22
  • 2
    $\begingroup$ As far as I can tell, for a general isomorphism between $V$ and $V^*$, it isn't necessarily the case that the induced bilinear form is symmetric or positive definite. $\endgroup$ – Daniel Schepler May 13 at 0:31
  • 1
    $\begingroup$ I think the map from $V$ to $V^*$ being an isomorphism is equivalent to the bilinear form being nonsingular (not sure if that's standard terminology, what I mean by that is $\langle v, w \rangle = 0$ for all $w$ implies $v = 0$ - that would fairly directly translate into the map $V \to V^*$ being injective, and then the two being vector spaces of equal finite dimension...). $\endgroup$ – Daniel Schepler May 13 at 19:03
7
$\begingroup$

This sort of question is why I object to using the term “dot product” as a synonym for the Euclidean inner product on $\mathbb R^n$, which is what I assume that you mean here. It’s important to distinguish the two, just as it’s important to distinguish between the elements $v$ of a vector space and their coordinates $[v]_{\mathcal B}$ relative to some ordered basis $\mathcal B$, especially when the vectors themselves are coordinate tuples.

Inner products are independent of the choice of basis. They are just functions that take a pair of vectors and spit out a scalar that have certain nice properties. However, the expression for an inner product $\langle\cdot,\cdot\rangle$ in terms of coordinates of those vectors is basis-dependent. Generally speaking, it’s not the case that $\langle v,w\rangle = [v]_{\mathcal B}^T[w]_{\mathcal B}$. In fact, this only holds when $\mathcal B$ is orthonormal relative to $\langle\cdot,\cdot\rangle$. The right-hand side of that expression is what I would call a “dot product:” it’s a specific computation involving a pair of $n\times 1$ matrices. Now, it happens that if the vectors are elements of $\mathbb R^n$ then the standard basis is orthonormal relative to the Euclidean inner product and their standard coordinate tuples are identical to the vectors themselves, so one can be somewhat cavalier about these distinctions in that context. In general, though, the coordinate formula for an inner product is going to be of the form $\langle v,w\rangle = [v]_{\mathcal B}^TG[w]_{\mathcal B}$ for some fixed symmetric matrix $G$ that’s determined by the inner product and $\mathcal B$. It’s a worthwhile exercise to work out what $G$ is in terms of change-of-basis matrices.

This holds no less in a subspace $V$ of an inner product space. The inner product is inherited from the parent space, and gives the same result regardless of whether we restrict our attention to $V$ or not. On the other hand, its expression in coordinates relative to some basis of $V$ again depends on the choice of basis: if the basis is orthonormal, then it’ll be equal to the dot product of the coordinate tuples, although those coordinate tuples will now be shorter than they were when considering the entire parent space.

There’s a similar condition for turning the application of a covector to a vector into a simple matrix multiplication: If $v\in V$ and $\phi\in V^*$ and we represent the coordinates of a covector as a $1\times n$ matrix, then $\phi(v)=[\phi]_{\mathcal B^*}[v]_{\mathcal B}$ iff the two bases are dual. That is, if $\mathcal B=(v_1,\dots,v_n)$ and $\mathcal B^*=(\beta_1,\dots,\beta_n)$, we must have $\phi_i(v_j)=\delta_{ij}$ for the above identity to hold. To bring this full circle, the Riesz representation theorem connects covectors and inner products: If $H$ is a Hilbert space, then for every element $\phi\in H^*$ there is some fixed $x\in H$ such that $\phi(y)=\langle y,x\rangle$ for all $y\in H$.


I should note that if we’re talking about complex vector spaces, then we need to take the conjugate transpose instead of a simple transpose, i.e., $[v]_{\mathcal B}^H[w]_{\mathcal B}$ instead of $[v]_{\mathcal B}^T[w]_{\mathcal B}$.

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

No. There is really nothing sacrosanct about a column vector as opposed to a row vector, for one thing.

For another, the dot product of two vectors is defined in a certain way, that makes sense. Namely $\vec a\cdot\vec b=a_1b_1+a_2b_2+\dots+a_nb_n$, where the $a_i$ and $b_i$ are the components of $a$ and $b$ respectively.

As long as you stick to the definition you won't run into trouble.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ How can I formalize this idea? That is, how does the process of representing my metric tensor $g$ by a $n\times n$ matrix that acts on two vectors represented (respectively) by a line and column vectors? $\endgroup$ – Syr May 12 at 0:25
  • $\begingroup$ Write the first as a row vector, and the second as a column vector. $\endgroup$ – Chris Custer May 12 at 0:57
  • $\begingroup$ I'm asking this because many books use the folowing notation: line vectors are vectors lying in $V$ and column vectors are 1-forms lying in $V^*$ $\endgroup$ – Syr May 12 at 1:01
  • $\begingroup$ Yes, that means you have to proceed differently. You can't do left to right multiplication. So try right to left. See how they handle the situation in the text, also. $\endgroup$ – Chris Custer May 12 at 1:05
3
$\begingroup$

No. You're confusing the dot product with matrix multiplication.

With the dot product you take two vectors and your final answer is one scalar (number) and the two vectors need to be of the same dimension because that's how the dot product was defined.

For matrix multiplication, you take two matrices and your final answer is another matrix (or a row vector (1xn matrix) or a column vector (nx1 matrix)), but for this you need the number of columns of your first matrix to equal the number of rows of your second matrix because that's how matrix multiplication was defined.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ "the number of columns of your first matrix to equal the number of rows of your second matrix". Right. And the dot product is done by multiplying two matrices, one is 1×n and the other is n×1, resulting in a 1×1 result (scalar). The OP is pointing out that to do this operation, one of the vectors must first be converted from n×1 to 1×n form. $\endgroup$ – Ray Butterworth May 12 at 0:30
  • 3
    $\begingroup$ They're not related by usual definitions, but they can be related. If we treat the set of column vectors as a vector space, we can notice (or define) the dot product $\langle v_1, v_2 \rangle = v_1^T v_2$, just using $\langle,\rangle$ instead of a dot to avoid confusion. Then you get interesting facts like $\langle v_1, A v_2 \rangle = \langle A^T v_1, v_2 \rangle $, etc. $\endgroup$ – aschepler May 12 at 0:33
  • $\begingroup$ @RayButterworth I never viewed the dot product this way, or a 1x1 matrix as a scalar. That's interesting. $\endgroup$ – Derpp May 12 at 0:35
1
$\begingroup$

The "dot product" of two vectors $v_{a}$ and $v_{b}$ is often expressed as a matrix multiplication, $$ v_{a} \cdot v_{b} = v_{a}^{T} v_{b}, $$ but can be written without matrix notation as the sum of the pairwise products of the vector components, $$ v_{a} \cdot v_{b} = \sum_{i} v_{a}^{i} v_{b}^{i}. $$

Similarly the matrix-weighted inner product is often expressed via matrix multiplication as $$ \langle v_{a} , v_{b}\rangle = v_{a}^{T} M v_{b}, $$ but can be written without matrix notation as $$ \langle v_{a} , v_{b}\rangle = \sum_{i,j} v_{a}^{i} M_{ij}\, v_{b}^{j}. $$

For both products, transposing $v_{a}$ and using the matrix operation is an implementation of the sum-over-index approach, but doesn't fundamentally change anything about the type of vector that you're working with.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

You say:

"...to take the dot product, you need to convert one of the vectors into a row vector which lives in a completely different dual subspace $V^*$ and then take the dot product of this dual space vector with the column vector. Is all of this true?"

Almost yes: it seems to me that the non 100% accurate part is "and then take the dot product of this dual space vector with the column vector".

It is matter of language and interpretation, but it may be convenient (especially in General Relativity) to think that the inner product is an operation that eats two vectors (what you call column vectors) and not a vector and a dual vector (in fact, see the standard definition).

Apart from this minor point, you are totally right: the inner product operation consists of creating a "row-vector" from a "column-vector". This "row-vector" lives in the dual space (and in many contexts it is regarded as a 1-form, or linear form): this linear form eats a vector and gives you a scalar.

To summarize, the idea behind the dot-product operation consists of three passages:

1) take two (column) vectors.

2) use a duality operation to construct the "covector", or "1-form" associated to one of those two vectors. There are many ways to do this (for example in General Relativity you don't make a transposition but you lower the index with the metric tensor.. and in Quantum Mechanics a vector "ket" becomes a "transposed" dual object called "bra", see e.g. this).

3) now, the linear form eats the other (row) vector and gives you a number, the output of the dot-product operation.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.