# How to represent matrix multiplication in tensor algebra?

How can we represent matrix multiplication in tensor algebra?

Even if we assume all matrices represent contravariant tensors only, clearly matrix multiplication does not correspond to the multiplication operation of the tensor algebra (the tensor product), since the former is grade-preserving or grade-reducing, whereas the latter is always grade-increasing.

And then if we allow matrices to represent either contravariant or covariant or mixed variance tensors, then things get even more confusing.

For instance, a quadratic form then can be represented by the same matrix as the bilinear form it generates via polarization.

Seemingly we must implicitly be using the universal property relating $V \otimes V$ (tensor product) and $V \times V$ (Cartesian product). But we can define the same type of (matrix) multiplication for $V \otimes V^*, V^* \otimes V,$ or $V^* \otimes V^*$ or between elements of any two.

Thus now even the claim that matrices represent linear transformations and that matrix multiplication is the composition of linear maps seems suspect to me.

Is this just a result of the fact that linear algebra was invented before multilinear algebra/tensor analysis, and thus people were abusing notation when using matrices without realizing it but then the convention stuck? Or is there something more to this which I am missing?

Related but more abstract and slightly different question: How do we describe standard matrix multiplication using tensor products?

Relevant wikipedia articles: https://en.wikipedia.org/wiki/Outer_product#Tensor_multiplication, https://en.wikipedia.org/wiki/Kronecker_product

• @MattSamuel Then why is the tensor product of two vectors equivalent to matrix multiplication, but matrix multiplication for arbitrary tensors bears no relationship with the tensor product whatsoever? en.wikipedia.org/wiki/Outer_product#Tensor_multiplication – Chill2Macht Jul 2 '16 at 6:30
• It's a little hard for me to figure out what you really want in the end. If you allow yourself the concepts of function composition and bases then there's no way around inventing matrix multiplication in the pedestrian way. If you must mix this up with tensors then maybe you could use the identification of $\operatorname{Hom}(V,W)$ with $V^* \otimes W$ and start contracting tensors, but my guess is that this is content-free. – Hoot Jul 2 '16 at 6:35

Fix a basis $\{e_1, \ldots, e_n\}$ of $V$, and consider the dual basis $\{f_1, \ldots, f_n \}$ of $V^\ast$. Then we have a basis $$\{e_1\otimes f_1,\ldots, e_i \otimes f_j, \ldots, e_n \otimes f_n\}$$ for $V \otimes V^\ast$, and the matrix $$A = (a_{ij})$$ is just a way of representing the element $$\sum_{i=1}^n \sum_{j=1}^n a_{ij} \; e_i \otimes f_j \in V \otimes V^\ast.$$

Of course an element of $V \otimes V^\ast$ gives a linear map $V \to V$ by

$$(w \otimes f)(v) := f(v) w$$

and extending by linearity. Given two such elements, we can compose the corresponding functions:

$$(w' \otimes f')(w \otimes f)(v) = (w' \otimes f')(f(v) w) = f(v) f'(w) w' = f'(w) \; (w' \otimes f)(v)$$

so composition of linear maps is given by

$$(w' \otimes f') \circ (w \otimes f) = f'(w) \; (w' \otimes f)$$

extended by linearity. If you write your elements in the $e_i \otimes f_j$ basis and apply this operation to them, you'll see that the usual definition of matrix multiplication pops right out.

Of course all the calculations with explicit tensors above can be rephrased in terms of the universal property of the tensor product if you like.

This is all assuming you want the matrix to represent an element of $V \otimes V^\ast$ rather than an element of $V \otimes V$ or $V^\ast \otimes V^\ast$. But you can work out what should happen in cases like that the same way.

• So if the matrix represented an element of $V^* \otimes V$ it would represent a linear map $V^* \to V^*$? What about $V \otimes V$? Would it represent a linear map $V \to V^**$ or $V^* \to V$? -- Also this a great answer by the way -- it deserves more votes. – Chill2Macht Jul 2 '16 at 15:46
• Since $V \otimes W$ is naturally isomorphic to $W \otimes V$, the order isn't really important. Elements of $V \otimes V^\ast$ naturally induce a map $V \to V$ and also a map $V^\ast \to V^\ast$. In some cases you'd fix a particular convention to make everything flow smoothly, and there are established choices in certain contexts, but choices like this are ultimately arbitrary, sort of like how everyone agrees to drive on either the left or right side of the road, depending on the country. – Daniel McLaury Jul 2 '16 at 15:51
• I guess it's worth noting that if you're working with an infinite dimensional vector space then $Hom(V^\ast, V^\ast) \cong V^\ast \otimes V^{\ast\ast}$. $V$ is naturally a subspace of $V^{\ast\ast}$, so elements of $V \otimes V^\ast$ give linear maps $V^\ast \to V^\ast$, but you only get every such map in this way if $V$ is finite dimensional, in which case $V^{\ast\ast} = V$. – Daniel McLaury Jul 2 '16 at 15:57