Is there a fluid connection going from a field like $\mathbb R$ to a vector space over $\mathbb R$ to a matrix to higher-order tensors? I'm trying to develop an intuition and motivation for tensors but it seems like vectors and matrices are treated very differently even though they are technically just order-1 and order-2 tensors.

Matrices are a representation of a linear map between vector spaces (or a linear operator within a vector space), is there a notion of vectors as some sort of map between fields or within a field?

Is a 3rd-order tensor a linear map between matrices (if we consider them as the objects of study)? I know a higher order tensor is considered a multilinear map but in some cases the objects of interest are not vectors but are matrices or higher order tensors.

E.g. if you're studying images as mathematical objects, it is natural to represent a grey-scale image as a matrix or an RGB image as a 3rd-order tensor. So do I use higher-order tensors to 'mutate' (map) between these objects, without evening thinking about vectors?

  • 1
    $\begingroup$ Matrices in fact represent several objects. They could represent a linear operator $V\to V$. But they could also represent a bilinear form $V\times V\to F$. In fact, the space of linear operators $\text{Hom}(V, V)$ is isomorphic to the space of bilinear maps $V\times V^{*}\to F$, where $V^{*}$ is the dual space. I'm glossing over important technicalities, but the point is that matrices really are order-2 tensors (viewed as multilinear maps). But a linear operator has both a covariant and a contravariant index, while a bilinear form has two covariant indices. $\endgroup$ Nov 14 '17 at 14:40

Well, first note that every continuous linear form in $\mathbb{R}^n \rightarrow \mathbb{R}$ is represented by a unique vector in $\mathbb{R}^n$ (this is the Riesz representation theorem), e.g., $\phi_v (x) = \langle v, x\rangle $ where $ v $ is the representer of $\phi_v$.

By the same token, you can of course think of scalars as representing (purely) linear functions: $f_{\alpha}(x) = \alpha x $.

As you have pointed out, matrices are usually thought of as representing linear transformations. However, note that this is not the only viewpoint: every $ m \times n $ matrix $ A $ defines a bilinear form in $ \mathbb{R}^m \times \mathbb{R}^n \rightarrow \mathbb{R} $ via $ A (x, y) = x^T A y $ . In fact, given choices of bases for these vector spaces, every matrix is uniquely associated with a bilinear form and vice-versa. When you pass to higher orders, you have essentially the same interpretation for hypermatrices, which represent multilinear transformations (tensors).

Now, note you don't need a tensor to define a map from one matrix space to another: a matrix will do! Namely, if you consider the natural isomorphism between the space of $ m \times n $ matrices and $\mathbb{R}^{mn} $, then any linear transformation from a matrix of this kind into another of dimensions $ p \times q $ can be represented by a $ pq \times mn $ matrix (essentially, you "vectorize" them).

Similarly, a bilinear form mapping a pair of matrices into a scalar can be represented by a matrix. However, if you want your bilinear map to yield a vector (or a matrix), then it must be represented by a hypermatrix. For instance, the matrix multiplication is a bilinear operation in $\mathbb{R}^{m \times n} \times \mathbb{R}^{n \times p} \rightarrow \mathbb{R}^{m \times p}$ and is associated with a tensor (often called Strassen tensor). Interestingly, the rank of this tensor quantifies the algebraic complexity of this operation. Indeed, Strassen's algorithm shows you can multiply $2 \times 2 $ matrices using 7 multiplications instead of 8, and this is possible precisely because the associated tensor has rank 7. (Similarly, the rank of a matrix quantifies the algebraic complexity of the associated bilinear form.)

Edit: See https://en.m.wikipedia.org/wiki/Strassen_algorithm?wprov=sfla1 for more information on Strassen's algorithm and its underlying tensor.


First of all, you can see a field $k$ as a vector space over itself. A vector can thus be with a linear map $k^n\to k$ or $k\to k^n$ since you can see it as a matrix of dimension $n\times 1$ or $1\times n$.

Example: $\left(\begin{array}{c} a \\ b \end{array}\right)$ can be identified with the linear map $c\mapsto \left(\begin{array}{c} ac \\ bc \end{array}\right)$.

Likewise $n\times n$-matrices represent maps from $k^n$ to itself.

But tensors are a bit more abstract. Let's suppose we have two vector spaces $V$ and $W$ over the same field $k$. Then we can form the product $V\times W$, but this is not particularly nice with respect to linearity of maps.

What we do instead is that we construct a certain vector space $V\otimes W$ which behaves well with respect to linearity in both arguments. In various areas of mathematics, the elements of $V\otimes W$ are called "tensors".

Example: For $V=W=k^n$, we get $V\otimes W=k^{n\cdot n}$.

Another approach is to consider "multilinear maps", aka tensors, between vector spaces, but the nice thing is that these naming conventions actually work together nicely.

I reccommend reading this: https://en.wikipedia.org/wiki/Tensor#As_multilinear_maps


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.