# What is the operation that happens between a tensor product and a vector under matrix multiplication?

So this question is fairly simple. Typically undergraduates are taught that matrices and tensors are their own thing and not truly interchangeable in a general sense. However, there is a detail that has bothered me for a minute. Displacement vectors and transformations and mappings being the prototypical examples.

Let $$A$$ be a matrix of any kind, singular or otherwise such that $$[A]_{ij}=A_{ij}$$ and $$[\vec{b}]_b=b_b$$ Matrix-Vector multiplication, $$A\vec{b}$$, can be written in einstein notation: $$[A\vec{b}]_i = A_{ij}b_j$$ or if we include the basis: $$A\color{red}{\cdot}\vec{b} = A_{ij}b_j\hat{e}_i\\ \color{red}{\text{the operation is definitely the dot product then}}$$

However there is a method of representing matricies in the form of linear combinations of tensor products of basis vectors. When we include the basis vectors in the definitions, we may omit the square brackets: $$A=A_{ij}\hat{e}_i\otimes\hat{e}_j\ \text{ and }\ \vec{b}=b_b\hat{e}_b$$

The matrix product between them still has the same components, but what operation is happening between $$\hat{e}_i\otimes\hat{e}_j$$ and $$\hat{e}_b$$? Is it literally just the dot product? \left(\hat{e}_i\otimes\hat{e}_j\right)\cdot\hat{e}_b \stackrel{?}{=} \hat{e}_i\otimes\left(\hat{e}_j\cdot\hat{e}_b\right) \stackrel{?}{=} \hat{e}_i\delta_{jb}\\ \Downarrow\\\begin{align*} A\vec{b} &=A\cdot\vec{b} = A_{ij}\hat{e}_i\otimes\hat{e}_j\ \cdot\ b_b\hat{e}_b = A_{ij}b_b\ \hat{e}_i\otimes\hat{e}_j\cdot \hat{e}_b \\ &= A_{ij}b_b\ \hat{e}_i\delta_{jb} = A_{ij}b_b\delta_{jb}\ \hat{e}_i = A_{ij}b_j\ \hat{e}_i \\ \end{align*}

I can't actually find an answer to this, or if I did see it somewhere it was years ago. I found this, but it's not the same, though this seems to be sufficiently similar to at least justify the associativity implied above. Is the tensor product with a scalar defined? $$\hat{e}_i\otimes\left(\hat{e}_j\ \cdot\ b_b\hat{e}_b\right) \stackrel{?}{=} \hat{e}_i\otimes\delta_{jb} \stackrel{?}{=} \hat{e}_i\delta_{jb}\\ \hat{e}_i\otimes a\stackrel{?}{=}a\hat{e}_i, a\in\mathbb{R}$$

• When you say that $\vec{c} = A \vec{b}$, you are acting with a $(1,1)$-tensor on a vector, i.e., an object that maps a vector and a co-vector to the reals: $A: V^* \times V \rightarrow \mathbb{R}$. So it should really be thought of in tensor notation as $\vec{c} = A\left(\cdot, \vec{b})\right)$. Given a basis of vectors, $\left\{ e_i\right\}$ and a basis of co-vectors, $\left\{E^i\right\}$, you can express the $(1,1)$-tensor $A$ in components, as $A = A^i_j \, e_i \otimes E^j$. Dec 1, 2022 at 2:32
• See, for example this section of Schuller's GR tutorials, where a $(1,1)$-tensor is used to create a map from a co-vector to a co-vector (instead of a map from a vector to a vector). And, of course, watch the associated video on Multilinear Algebra by Schuller. Dec 1, 2022 at 2:45
• So thank you for answering the first question. If you answer the second about the tensor product between a rank $(0,1)$ tensor and a $(0,0)$ I will mark it as the answer. Dec 1, 2022 at 3:07
• I'll try to write up a few things as an answer. Dec 1, 2022 at 3:08

I would recommend Schuller's lecture on Multilinear Algebra for a very nice from-the-ground-up description of vector spaces and tensors, but it doesn't include a discussion of the tensor product. Lee's Riemannian Manifolds has a very brief review of tensors in Chapter 2, which does include tensor products. The essentials are:

• Given a $$d$$-dimensional vector space $$V$$ over the reals, the dual space of covectors $$V^*$$ is the set of all linear maps from the vector space to the reals, i.e., $$\omega \in V^*$$ means: \begin{align} \omega: V &\rightarrow \mathbb{R}\\ v &\mapsto \omega(v) \in \mathbb{R} \end{align}

• An $$(l, k)$$-tensor is a linear map from $$l$$ copies of $$V^*$$ and $$k$$ copies of $$V$$ into the reals, e.g., a $$(1,1)$$-tensor is \begin{align} T: V^* \times V &\rightarrow \mathbb{R}\\ (\omega,v) & \mapsto T(\omega, v) \in \mathbb{R} \end{align} The elements of $$V$$ (vectors) are also elements of $$\left(V^*\right)^*$$, so $$v\in V$$ means that it is a map: \begin{align} v: V^* &\rightarrow \mathbb{R}\\ \omega &\mapsto v(\omega) \in \mathbb{R} \end{align} Covectors are $$(0,1)$$-tensors and vectors are $$(1,0)$$-tensors. Their pairing to give a real number, e.g., $$v(\omega)$$ and $$\omega(v)$$, is the multiplication of a row and column vector in elementary linear algebra.

• Given a basis $$\left(e_1, \ldots e_d \right)$$ of $$V$$, the corresponding dual basis of $$V^*$$ is $$\left(E^1, \ldots, E^d\right)$$, where $$E^i\left(e_j\right) = \delta^i_j \quad \forall\; i,j$$

• The components of a tensor in a chosen basis of $$V$$ are found by its actions on the basis vectors and corresponding dual basis covectors. So for the $$(1,1)$$-tensor above, $$T\left(E^i, e_j\right) =: T^i_j$$

• We can express the tensor in that basis via the tensor product $$T = T^i_j \; e_i \otimes E^j$$ where summation is implied over repeated (up-down) index pairs, and the set of tensor products $$\left\{e_i \otimes E^j\right\}$$ are the basis elements for $$(1,1)$$ tensors.

• The action of that tensor on an arbitrary co-vector/vector pair, $$(\omega, v)$$, is then written: $$T(\omega,v) = T^i_j \; \left[e_i \otimes E^j\right](\omega,v) = T^i_j \; e_i(\omega) E^j(v) = T^i_j \;\; e_i\left(\omega_k E^k\right) E^j\left(v^m e_m\right) = T^i_j \, \omega_i \, v^j$$ where we have expanded $$v$$ and $$\omega$$ in their respective bases, used the linearity of the basis vectors and covectors, and then used the defining relationship between the basis vectors and the corresponding dual basis covectors.

So, in your question, when you write $$\vec{c} = A \vec{b}$$, you are really acting on the vector $$\vec{b}$$ with a $$(1,1)$$-tensor, leaving it with one open "spot", resulting in a $$(1,0)$$ tensor, i.e., a map from $$V^*$$ to the reals, otherwise know as a vector: $$A\left(\cdot, \vec{b}\right) \in V$$ Such a map from $$V$$ to $$V$$, allowed by a $$(1,1)$$ tensor, is called an endomorphism. Following the prescription above, we could write $$A$$ as: $$A = A^i_j \; e_i \otimes E^j$$ and its action on $$\vec{b}$$ as $$A(\cdot,\vec{b}) = A^i_j \; e_i(\cdot) E_j(\vec{b}) = A^i_j \, b^j \, e_i(\cdot)$$

You can think of other rank-2 tensors as having components in matrix form, e.g., the inner product of two vectors: $$g: V \times V \rightarrow \mathbb{R} \quad \text{with} \quad g = g_{ij} \; E^i \otimes E^j$$ but only a $$(1,1)$$-tensor's matrix of components executes the familiar matrix multiplication of elementary linear algebra. (When $$g$$ acts on a single vector it still has an open spot for another vector, so $$g(\cdot, v)$$ is a covector, and therefore $$\left[g_{ij}\right]$$ should just be thought of as a convenient way of packaging the components of $$g$$. But we could use $$g$$ to map from $$V$$ to $$V^*$$, converting a column vector to a row vector, as is done with Dirac's bra-ket notation and the "raising/lowering" of indices in GR.)

As to your last question, about the tensor product of a vector and a scalar... You can see above from the second equality following "The action of that tensor" how a tensor product works: it is an ordered tuple of vectors and covectors, waiting for a same-ordered tuple of covectors and vectors to act on, and then you just multiply the resulting real numbers together. But a scalar is not a tensor since it eats no vectors or covectors (I guess you could think of it trivially as a "$$0$$-tensor"?), and therefore there would be nothing to take a tensor product with. I don't think the question has meaning, but I haven't thought about it carefully.

• $e_i\otimes E^j$ is a (1,1) tensor, but isn't $E_i\otimes E_j$ a (0,2 tensor) and $e_i \otimes e_j$ a (2,0) tensor? Dec 1, 2022 at 4:36
• Yes, although, you would sum over or specify the values of $i$ and $j$ to make them a tensor. A generic $(0,2)$ tensor is the sum over the basis pair and the tensor component, as is written for $g$ in my answer. Dec 1, 2022 at 4:43
• An inner product is defined to be $(0,2)$-tensor (often with a few other criteria, such as symmetry and positive-definite). So an inner product always takes two vectors. You can talk about the tensor product of two tensors, which just means that the resulting tensor takes an arranged order tuple of their arguments. The tensor product of an $(a,b)$ tensor and a $(c,d)$ tensor is a $(a+c, b+d)$-tensor; that is just a generalization of what is written above for the tensor product of vectors and covectors (see the linked text by Lee, bottom of second page, pg 12). Dec 1, 2022 at 4:57
• The dot product is another term for the inner product, so it is the action of a $(0,2)$-tensor on two vectors. Matrix multiplication is the the application of a $(1,1)$-tensor on a single vector to get back a vector. Dec 1, 2022 at 5:19
• Yes, that is true, although you should write it as $e_j\left(E^i\right) = \delta^i_j$, since these $e_j$ and $E^i$ are actual vectors and covectors, not components; they act on one another rather than being a product of numbers. But these statements --- like: there exists a unique corresponding dual basis of covectors that satisfies that relation, and that the "inverse" relation that you asked about is true --- are things that should be proven. See, e.g., in Halmos Finite-Dimensional Vector Spaces. Dec 1, 2022 at 19:08