I have some perplexities when I reading references about tensor product.

My main question is:

How to define the tensor product between two vectors?

It is clearly to define the tensor product of two column vectors as $v_1\otimes v_2=v_1v_2^\top$ which is according to matrix multiply (see Matrix multiply).

What is more, to define the tensor product of two linear maps, we have: $$ (A_1\otimes A_2)(v_1\otimes v_2)=w_1\otimes w_2 \qquad(1) $$ where $A_1 v_1=w_1$ and $A_2 v_2=w_2$ (see tensor product). When $A_1$ and $A_2$ are maps between column vectors, $(A_1\otimes A_2)$ can be written as Kronecker product.

However, these definitions confuse me. For example, let $v_1$,$v_2$, $w_1$ and $w_2$ are $2\times1$ column vectors and then $A_1$ and $A_2$ are $2\times$ matrices. They satisfy $A_1 v_1=w_1$ and $A_2 v_2=w_2$. Obviously, the Kronecker product $A_1\otimes A_2$ is a $4\times4$ matrix while $v_1\otimes v_2$ and $w_1\otimes w_2$ are all $2\times2$ matrices. Then $(A_1\otimes A_2)(v_1\otimes v_2)$ is meaningless because we can not calculate the multiply of $4\times4$ matrix and $2\times 2$ matirx ! So how to check $(1)$?

In 'Alexander Graham, Kronecker Products and Matrix Calculus: with Applications', the author check $(1)$ by define $v_1\otimes v_2$ and $w_1\otimes w_2$ as Kronecker product which is different from $v_1v_2^\top$.

So, as mentioned in beginning, how to define the tensor product between two vectors? Is there do not exist a unique definition about it? Or there is some points I missed?

Thanks for help!


There is no canonical way of having $4\times4$ matrices act on $2\times2$ matrices, but notice the space of $2\times2$ matrices is four-dimensional, and linear transformations of this space can be encoded as - you guessed it - $4\times4$ matrices. Write down the relations $(e_{ab}\otimes e_{cd})(e_u\otimes e_v)=\delta_{bu}\delta_{dv}(e_a\otimes e_c)$, where $e_{ij}$ are the elementary matrices and $e_k$ the obvious coordinate vectors, then convert everything on both sides to the relevant matrices via Kronecker products. This tells you how to apply elementary $4\times4$ matrices to elementary $2\times2$ matrices; everything else follows from using the distributive property (write matrices as sums of elementary matrices).

Tensors of vectors can be thought of in different ways. One way is as multidimensional arrays of numbers (usual vectors are one-dimensional arrays, matrices are two-dimensional, etc.), possibly with the "axes" of this array partitioned into upper or lower indices, or covariant versus contra-variant dimensions. This would be highly relevant in physics.

To a pure mathematician, $V\otimes W$ is first and foremost defined implicitly via universal properties, which comes with a canonical explicit construction in terms of the "symbols" $v\otimes w$ which satisfy bilinearity relations. This is my preferred method, as it captures the fact we're trying to left-multiply elements of $V$ against elements of $W$, make scalars in the base field commute with everything, and impose a distributive property.

Finally, as vector spaces there is an isomorphism $K^n\otimes_KK^m\cong M_{n\times m}(K)$ which takes pure tensors of coordinate vectors to their Kronecker product. I suppose this might be useful computationally in some settings, but I am not used to this method.

  • $\begingroup$ Thank you for your answer. Did you mean that $v_1v_2^\top$ and Kronecker product of them are equivalent by reshape them? $\endgroup$ – Lion Jun 1 '14 at 9:21
  • $\begingroup$ @Lion It's true that $v_1v_2^\top$ is the Kronecker product of $v_1$ and $v_2^\top$, and that I was speaking of $v\otimes w\mapsto vw^\top$ for the isomorphism $K^n\otimes_KK^m\cong M_{n\times m}(K)$. $\endgroup$ – blue Jun 1 '14 at 17:33
  • $\begingroup$ OK, Thank you! But could you help me for another question: what is the meaning of the algebraic definition of tensor product: $F(X\times Y)/R$ which given by en.wikipedia.org/wiki/Tensor_product I can not imagine what the definition describe? And why $(a,b)+(c,d)=(a+c,b+d)$ is invalid in free vector space? Thank you for your help again! $\endgroup$ – Lion Jun 2 '14 at 0:33
  • 1
    $\begingroup$ @Lion The free vector space is free on the set of coordinate vectors, so the vectors $(a,b)$, $(c,d)$ and $(a+c,b+d)$ are all linearly independent by definition. Do you know what free vector space means? And $(a,b)+(c,d)=(a+c,b+d)$ fails in the quotient too. That is a relation in the direct sum of vector spaces, not tensor product of them. The tensor symbol is meant to function as multiplication, which obeys distributivity on both sides (bilinearity). Do you think $ab+cd=(a+c)(b+d)$ is true for numbers $a,b,c,d$? No. $(a+b)x=ax+bx$ and $y(c+d)=yc+yd$ are true though. $\endgroup$ – blue Jun 2 '14 at 0:40
  • 1
    $\begingroup$ The point of the construction is to allow us to left-multiply vectors of $V$ with vectors of $W$. If $v\in V$ and $w\in W$ then denote their "product" by the symbol $v\otimes w$. We want multiplication to be distributive, so we have $(a+b)\otimes x=a\otimes x+b\otimes x$ and $y\otimes(c+d)=y\otimes c+y\otimes d$ for all $a,b,y\in V$, $c,d,x\in W$. We want scalar multiplication to work as usual, so $(\lambda v)\otimes w=v\otimes(\lambda w)$ for all scalars $\lambda$ and $v\in V$, $w\in W$. We then allow ourselves to make linear combinations of these products $v\otimes w$. We get $V\otimes W$. $\endgroup$ – blue Jun 2 '14 at 0:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.