# Tensor product in multilinear algebra

In the book by Halmos ($FDVS$) the tensor product of two vector spaces U and V is defined as the dual of the vector space of all the bilinear forms on the direct sum of U and V. Is there a generalised form of this for the direct sums of more than two vector spaces? Is there a relation between the space of all multilinear forms on the direct sum of $V_1$,$V_2$,$V_3$,...,$V_k$ with their tensor product.

Please explain without invoking other algebraic objects such as modules,rings etc and by using the concepts regarding vector spaces only (as the book assumes no such background either, it is unlikely that any reader of that book will benefit from such an exposition). Everywhere I searched, I found the explanation in terms of those concepts only and being unfamiliar to those I couldn't get them at all. Thanks in advance.

• It's understandable that you'd desire an explanation in terms of "simpler" things, as you mention in reaction to the (good) answers, but (as @RyanReich notes) "tensor products" are typically difficult for beginners... In part, this is due to their genuinely more sophisticated nature. In some formal symbol-pushing sense they can be explained in "more elementary terms", but such an explanation will not really be helpful! Instead, the "issue" of tensor products is a suggestion and invitation to begin taking a more sophisticated viewpoint. Diagrams? Universal mapping properties? – paul garrett Jun 3 '14 at 16:44
• @paulgarrett : Honesty speaking I have very little background other than Halmos. I am willing to learn the new viewpoint if that is what it takes but the diversity of options to do so is overwhelming. If there is a certain definition that can state the essence of the matter then I would be really glad to learn it. – O_huck Jun 4 '14 at 12:03
• Halmos' definition is not really good ... – Martin Brandenburg Jun 4 '14 at 15:55

What is a bilinear form? Naively, it is a certain kind of set-function $\phi\colon V\times W\to \Bbbk$ (where $\Bbbk$ is the base field over which our vector spaces are defined). More cleanly, however, one should think of a bilinear form as an element of the vector space $\DeclareMathOperator{\Hom}{Hom}\Hom_\Bbbk(V,\Hom_\Bbbk (W,\Bbbk))$ where $\Hom_\Bbbk$ is the space of $\Bbbk$-linear (i.e. linear with respect to the scalars in the field $\Bbbk$) maps between vector spaces.

This expresses the fact that evaluating a bilinear from $\phi(v,w)$ in the first variable (plugging in something for $v$) should give a function $\phi(v,-)\colon W\to\Bbbk$ that is linear. So more concisely, we can say that a bilinear form is a linear map in $\Hom_\Bbbk(V,\Hom_\Bbbk(W,\Bbbk))=\Hom_\Bbbk(V,W^*)$, i.e. a linear map from $V$ to the dual of $W$.

What is the tensor product? The tensor product $V\otimes W$ is a gadget which aims to represent bilinear maps as linear maps on a domain which depends naturally on $V$ and $W$. In other words, the tensor product $V\otimes W$ should satisfy the property that $\Hom_\Bbbk(V\otimes W,\Bbbk)\cong\Hom_\Bbbk(V,\Hom(W,\Bbbk))$, i.e. that $(V\otimes W)^*\cong\Hom_\Bbbk(V,W^*)$.

Now, recalling that a vector space $Z$ embeds in its double dual $Z^{**}$, we obtain $V\otimes W\hookrightarrow (V\otimes W)^{**}\cong\Hom_k(V,W^*)^*$. It follows that the tensor product is always contained in the dual of bilinear forms. When $V$ and $W$ are finite-dimensional, then $W^*$ is finite-dimensional, so $\Hom_\Bbbk(V,W^*)\cong (V\otimes W)^*$ is finite-dimensional, so $\Hom_\Bbbk(V,W^*)^*\cong (V\otimes W)^{**}$ is and thus is isomorphic to $V\otimes W$.

To generalize this to multi-linear forms, note that a multilinear form $\phi\colon V_1\times V_2\times\dots\times V_n\to\Bbbk$ should be thought of as an element of the vector space $\Hom_\Bbbk(V_1,\Hom_\Bbbk(V_2,\dots,\Hom_\Bbbk(V_n,\Bbbk))\dots)=\Hom_\Bbbk(V_1,\Hom_\Bbbk(V_2,\dots,\Hom_\Bbbk(V_{n-1},V_n^*))\dots)$. This should be naturally isomorphic to $\Hom_\Bbbk(V_1\otimes V_2\otimes\dots\otimes V_n,\Bbbk)=(V_1\otimes\dots\otimes V_n)^*$, so again $(V_1\otimes\dots\otimes V_n)^{**}\cong\Hom_\Bbbk(V_1,\Hom_\Bbbk(V_2,\dots,\Hom_\Bbbk(V_{n-1},V_n^*)\dots)^*$, i.e. again the double dualr of the multi-variate tensor product should be naturally isomorphic to the dual of the space of multi-linear forms. When the latter is finite-dimensional (if all the $V_i$ are finite-dimensional), then so is its dual, hence so is the double dual of the tensor product.

(There is nothing weird about the fact that the tensor product embeds in the dual of multi-linear forms. Taking a basis vector $v_1\otimes v_2\dots\otimes v_n$, by definition one can evaluate a multi-linear function $\phi\colon V_1\times\dots\times V_n\to\Bbbk$ at it by doing $\phi(v_1,v_2,\dots,v_n)$).

1. The tensor product $V\otimes W$ of $n$-dimensional $V$ with $m$-dimensional $W$ is (according to the above) an $n\cdot m$-dimensional vector space. More precisely, however, the tensor product $V\otimes W$ is an $n\cdot m$-dimensional vector space equipped with additional structure. This additional structure consists of a single piece of data: a designated isomorphism of the dual of the tensor product $(V\otimes W)^*$ with $\Hom_\Bbbk(V,W^*)$. This means that it does not actually matter which $n\cdot m$-dimensional vector space you consider to be the tensor product, what matters is the additional structure you put on it (or rather on its dual). So in fact every $n\cdot m$-dimensional space can be given the structure of being the tensor product $V\otimes W$, and then there are isomorphisms that preserve this structure, so up to isomorphism there is a unique tensor product.

2. Compare to how an inner product space is a vector space $V$ with a positive-definite symmetric bilinear form $\phi(-,-)\colon V\times V\to\Bbbk$. Being a bilinear form means that this is really a map in $\Hom_\Bbbk(V,V^*)$, and the other properties (positive-definiteness, symmetry) are properties of that map. Choosing different inner products gives different inner product structures (e.g. a positive weighted dot product versus the usual dot product on $\mathbb R^n$). Nevertheless, all inner product spaces of the same dimension are not only isomorphic as vector spaces, but the isomorphisms can be chosen to preserve the extra structure of the inner product, i.e. they are isometric. So again, up to isomorphism, there is only one inner product space of any particular dimension.

• Fantastic answer. I had never considered multilinear maps as you characterize them in your first two paragraphs. – Richard D. James Jun 3 '14 at 17:15
• "...the vector space $\DeclareMathOperator{\Hom}{Hom}\Hom_\Bbbk(V,\Hom_\Bbbk (W,\Bbbk))$ where $\Hom_\Bbbk$ is the space of $\Bbbk$-linear maps between vector spaces." Shouldn't k be 2 if we are considering the bilinear form. Thinking of it in terms of multilinear form makes perfect sense though. The above interpretation came up in some theorem I read but I never tried to think of the it in that way as a definition. Naively speaking, one more thing - why do we always define the tensor product as something that should have a certain property than to state what the vector space really is? – O_huck Jun 4 '14 at 11:51
• I understand that tensor product is an advanced topic and even in Halmos a section was devoted just for the motivation part but I failed to reconcile the definition he gave later on, with the motivation given earlier. What seems to to against intuition is the fact that whether the properties that 'should be' of the tensor product is not duplicated by another space of similar properties like dimension,etc. And so how do we know if what we are talking about is a well defined space? – O_huck Jun 4 '14 at 11:58
• I've tried answering your questions. $\Bbbk$ is the base field of the vector space. – Vladimir Sotirov Jun 4 '14 at 15:25
• Thanks for the help. Uniqueness up to isomorphism... I think I understand now. – O_huck Jun 6 '14 at 11:31

First of all: yes: just replace "bilinear" with "multilinear" and you have a definition that works for finite-dimensional vector spaces.

Second: as Halmos also says right there, this is not the correct definition for vector spaces in general. The reason is that the correct definition is actually indirect: it says that the dual of the tensor product is isomorphic to the space of multilinear forms on the direct sum. For FDVS's, this is equivalent to what he wrote, because the double dual of an FDVS is itself, but for infinite-dimensional spaces, this is not so.

Of course, for FDVS's you have the unsatisfying definition that "the tensor product of a $m$-dimensional and an $n$-dimensional space is an $mn$-dimensional space", since they are all determined completely by their dimensions. But that's not what's important in tensor products: what matters is the relationships that exist between vector spaces. And the relevant relationship for tensor products is:

For any vector spaces $V, W$ and $U$, there is a natural correspondence (in a sense that can be made precise using ideas you asked not to use) between linear maps $V \otimes W \to U$ and bilinear maps on $V \times W$ valued in $U$.

If we used the 1-dimensional space for $U$, we would recover the definition of $V \otimes W$ as the dual of the space of bilinear forms, but the above text is the real definition.

• By V×W do you mean the cartesian product of V and W? And does this holds for any U? Can you try to make it precise keeping the ideas outside Halmos to a minimum? – O_huck Jun 3 '14 at 16:07
• Yes, $V \times W$ is the cartesian product, and is identical as a set to $V \oplus W$. It does hold for any $U$. As for how to make it precise: the first section of the Wikipedia article "Tensor product" says what I would say. There's no simpler way to say it. – Ryan Reich Jun 3 '14 at 16:14
• Well I tried that but there seems to be too much new things to handle.Anyway thanks. – O_huck Jun 3 '14 at 16:20
• Tensor product is tied with the quotient construction for "most difficult idea for a beginner to understand about linear algebra". So, don't worry that there's no easy definition. – Ryan Reich Jun 3 '14 at 16:22
• @Andreas For FDVS's, sure. Halmos was a renowned teacher and his instinct here was good: his definition of the tensor product almost conveys the essence of the universal property. It's a fine introduction, though it doesn't leave much room for expansion: if you want to go any farther with the concept, you have to back up and start entirely from scratch. – Ryan Reich Jun 3 '14 at 19:25

If it's tensor products of more than two vector spaces, and not direct sums, yes, it's exactly the same characterization. (Usually a bilinear form is defined as a map $U \times V \to F$, where $F$ is the underlying field, which is linear in both arguments. But $U \times V$ is taken as a set, without the structure of a direct sum.)

That is, you can define the tensor product $V_{1} \otimes V_{2} \otimes \dots \otimes V_{k}$ as the dual of the space of all multilinear forms on $V_{1}, V_{2}, \dots , V_{k}$. The reason is that you want a universal property to hold so that there is an isomorphism of vector spaces between

• the space of multilinear forms, and
• the space $T^{\star}$ of linear forms on the tensor product $T$.

So take the dual, and note that in the finite-dimensional case $T^{\star \star} \cong T$.

• Oh sorry, the direct sum part was a typo. – O_huck Jun 3 '14 at 16:08

(As in my comment at the top...) Sincerely and seriously, apart from whatever short-term adaptations may be necessary or desirable... As @RyanReich's answer and comments suggest, an "elementary" "definition" of tensor product will leave a person mystified, for one thing. For another, it's completely unclear why we need such a thing. (Be thankful you aren't studying the late-nineteenth-century version of linear and multi-linear algebra, with "upper and lower multi-indices", thus spawning "covectors" $v^i$, which somehow were different from "vectors" $v_i$. "Different"? In what way? Well, supposedly, in the "rules"... Nevermind.)

The long-term advantageous viewpoint, which is much better for far more reasons than just understanding tensor products per se, has two parts. The first is about the transition from "definition" in the sense of "construction" (usually in set-theoretic terms), to "characterization" in the sense of how the thing is intended/required to interact with other things. Yes, this inverts the lower-level textbook-y approach of the last several decades, in which we (somewhat disingenuously!) give a "definition" at the very beginning ... not admitting that people went through hosts of examples and many trial definition before hitting on the best abstraction ... and then "proving" the (desired/required) properties in an exaggeratedly formal (but stylish) style. In particular, for tensor products, while one has been conditioned to expect "a construction", in truth that's not what we truly want: we want to know what the point is, what's going to be accomplished. But a certain overly-formal textbook version of mathematics has scant way to say this, except for... (drum-roll)...

"Category theory". Informal/naive category theory is a wonderful thing, in the same way that informal/naive set theory is a wonderful thing, as it gives a language and family of concepts that enrich us, that broaden the possibilities, show commonalities, etc. One must be cautious about the more "formal/axiomatic" versions, however, at least that they do tend to be "X for its own sake" or "axiomatizations of tangible things..." E.g., 100 years ago it was indeed interesting to hear from Kuratowski, Wiener, and others, that "an ordered pair" $(a,b)$ could be modeled "in set theory" (wherein "everything's a set") as $\{\{a\}, \{a,b\}\}$. But, srsly, folks, it is the characterization of "ordered pairs", namely, that $(a,b)=(a',b')$ if an only if $a=a'$ and $b=b'$ that we care about. That is, we care about the interaction of "an ordered pair" with the individuals "in" it, and with other ordered pairs, not the construction... although it's nice to know that clever people can model ordered pairs as sets.

Second, ... and here's a not-so-elementary point... a good "external" characterization of a thing often so-completely characterizes it that, by the very characterization (if done right), any two things $X,Y$ that do the job have a (unique) isomorphism between them (that also fits perfectly with whatever job is required). At a point where one distinguishes "equality" from "isomorphism" (which is sometimes apt, sometimes not), it is noteworthy that this does not say $X=Y$, but only that they are (uniquely) isomorphic. In truth, mostly this is the most one can reasonably demand. Often, this part follows _prior_to_ any construction! By the characterization! To repeat: although this is somewhat alien to elementary mathematics (typical undergrad?), the construction is not of interest, the interaction (as external characterization) is the point.

For example, with more-familiar objects: a quotient group $Q=G/N$ is a group $Q$ and (quotient) homomorphism $q:G\to Q$ such that any group hom $F:G\to H$ with $N$ inside its kernel "factors through $Q$, in the sense that there is $f:Q\to H$ such that $F=f\circ q$. Drawing a picture is nice, here, too.

What is an "indeterminate" $x$, as in the case of a polynomial ring $k[x]$ over a field $k$? We think of "variable" things, quite reasonable, but, seriously, what can this mean? It means that, for example, given a commutative ring $R$ with $k$ inside it, and given $a\in R$, there is a unique ("$k$-algebra") homomorphism $k[x]\to R$ sending $x$ to $a$.

For tensor products: a tensor product $V\otimes W$ of two $k$-vectorspaces is a $k$-vectorspace and bilinear $b:V\times W\to V\otimes W$ such that, for every bilinear $\beta:V\times W\to X$ for $k$-vectorspace $X$, there is a unique $k$-linear map $B:V\otimes W\to X$ through which $\beta$ factors, by $\beta=B\circ b$. From this characterization (not construction) all the relevant things can be proven. Does such a thing exist? Well, yes, and there are various (not so useful, really) constructions.

Part of the issue is that we are often given "the definition" of tensor products without any persuasive argument for caring about such things... and, then, the fact that they are somewhat more sophisticated things makes it doubly confusing.

So, that was the five-minute version of something. :)

• Thanks! I see the point now. – O_huck Jun 6 '14 at 11:27
• @paulgarret I have a question about your characterization of a tensor product. You say the tensor product $V \otimes W$ is a vector space associated with a bilinear $b$ which maps elements of $V\times W$ to elements of $V \otimes W$. And you say that bilinear $b$ is special, because for any bilinear functional $\beta$ over $V$ and $W$ it allows us to find a unique, associated linear functional on $V \oplus W$. This is promising, as it suggests a link (possibly a homomorphism??) between the space of bilinear functionals on $V$ and $W$ and the space of linear functionals on $V\otimes W$. – tom Apr 6 '15 at 14:08
• @paulgarret However, you then go on to say that the link between $B$ and $\beta$ is that $B$ is the unique linear functional on $V\oplus W$ with the property that '$b = B \otimes \beta$'. I'm totally lost here. You've used $\otimes$ to signify a tensor product between two vector spaces, but $B$ and $\beta$ are linear and bilinear functionals respectively. I just don't know how to interpret what you mean by $B\otimes \beta$. Thanks – tom Apr 6 '15 at 14:12
• @tom, oops, unfortunate typo: should have been "compose", not "tensor". I edited it. Also, it's not really a map on $V\oplus W$, but on $V\times W$, the cartesian product, not direct sum. – paul garrett Apr 6 '15 at 14:25
• @paulgarrett Sorry about the $\oplus$ - I meant $\otimes$ actually - $B$ is a linear functional on $V\otimes W$, or more generally a linear map $V\otimes W \rightarrow X$. I'm still a little unsure now about how that composition works. If $\beta$ maps from $V\times W \rightarrow X$, how can we compose it with $B$ which maps from $V\otimes W \rightarrow X$? Wouldn't we need $B: X \rightarrow V\otimes W$ so that $b(x,y) = B(\beta(x,y)) \in V\otimes W$? – tom Apr 6 '15 at 14:39