# Kronecker product and outer product confusion

I have two column vectors:

$$u = \left[\matrix{ 1 \cr 2\cr }\right]$$

$$v = \left[\matrix{ 4 \cr 4\cr }\right]$$

I'm trying to compute the Kronecker product of two vectors $u \otimes v$.

As I understand, the outer product of vectors is a special case of the Kronecker product of matrices.

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product $A \otimes\ B$ is the mp × nq block matrix.

So will $u \otimes v$ be of dimension 4 × 1 (according to the first definition) or 2 × 2 (according to the second definition)?

• The product $u \otimes v$ should be 4-by-1 if you follow the Wikipedia article on the Kronecker product strictly, but it should be 2-by-2 if you follow the Wikipedia article on the outer product strictly. This inconsistency is nothing serious. Both matrices have the same entries; they are just arranged differently. You should pick one that suits the way you're going to use the result. May 16 '14 at 4:30

This is a very good example of abuse of notation, more precisely, reload of operator. Actually the operator $\otimes$ is usually used as tensor product, which is a bilinear operator. It's easy to verify that both Kronecker product (denoted by $\otimes_K$) and outer product (denoted by $\otimes_O$) are bilinear and special forms of tensor product. For example, given two vectors $u,v\in V$, we have $$u\otimes_O v=u\otimes_Kv^H$$ This is why wiki says outer product is a special case of Kronecter product.

• Thanks, (I'd upvote if I could), I guess this explains the slide 7 of ima.umn.edu/industrial/2006-2007/kolda/kolda.pdf that says Observe: For two vectors a and b, $a\otimes_O b$ and $a\otimes_K b$ have the same elements, but one is shaped into a matrix and the other into a vector. May 16 '14 at 15:45
• another related question: Kruskal Tensor: sum of outer or Kronecker products?. If you have some time to check it out that would be very helpful. Thanks! May 16 '14 at 17:14

The relation between outer product $$\circ$$ and kronecker product $$\otimes$$ . $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} = vec(({\textbf{a}}\circ {\textbf{b}})^T) = vec(({\textbf{a}}{\textbf{b}^T}_{I \times J})^T_{J \times I})_{JI \times 1}$$

Before taking vectorization you should do a transpose.

In case of 3 vectors, the resulted matrix as before is multiplied by each entry in c vector and each result is a frontal slice of a tensor. Notice that $$.*$$ is an element wise product between each element in $$\textbf{c}$$ vector and the resultant matrix of $$(\textbf{a} \circ \textbf{b})$$ and not the element wise product between the whole vector $$\textbf{c}$$ and $$(\textbf{a} \circ \textbf{b})$$. $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} \otimes {\textbf{c}}_{K \times 1} = vec(( ({\textbf{a}}\circ {\textbf{b}}) .* {\textbf{c}})^T_{I \times J \times K})_{IJK \times 1}$$

Outer product between 2, each of 2 dimension then the result would be 4 dimension tensor which is totally different than the kronecker product

Kronecker Product for matrices $${\textbf{A}}_{I \times R} \otimes {\textbf{B}}_{ J \times R} = {\textbf{D}}_{ IJ \times R^2}$$

$${\textbf{D}} = \begin{bmatrix} \vdots & \vdots & & \vdots & & \vdots \\ \textbf{a}_1 \otimes {\textbf{b}_1} & {\textbf{a}_1} \otimes {\textbf{b}_2} & \dots & \textbf{a}_1 \otimes {\textbf{b}_R} & \dots &\textbf{a}_R \otimes {\textbf{b}_R} \\ \vdots & \vdots & & \vdots & & \vdots \end{bmatrix}$$

Outer Product for matrices $$\textbf{A}_{I \times R_1} \circ \textbf{B}_{J \times R_2} = {\textbf{C}}_{I \times J \times R_1 \times R_2}$$ to show how the result would be you have to do it column by column outer product, $${\textbf{a}_1} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{y}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{a}_1} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{n}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{p} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{j} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix},$$

The final result would be, $${\textbf{C}}_{I \times J \times 1 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{p}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{y} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 1 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{n} & {\textbf{j}} \\ \vdots & \vdots \end{bmatrix}$$

• What is the symbol $.*$ that you use? Elementwise product? If that is the case, the dimensions does not seem to add upp. Are you thinking about broadcasting like in some programming language? May 27 '20 at 13:47
• it is not element wise product with the vector c, instead it is element wise product with each element in that vector, each element in c is multiplied by the whole matrix of (aob) and this is one slice as the frontal slice for k=1 and at the end the result is a tensor of IxJxK size as c vector is of size K. Actually, I was working on my master thesis when I answered this question and currently, I am learning OOP programming creating User interfaces, I hope that I can utilize what I have learned here and there. Jun 5 '20 at 8:31
• Could you please consider describing this in the answer, or adding a reference to some literature, so that the answer is complete? Jun 5 '20 at 8:35
• I actually found no literature to describe this and this is how I found the question, I actually found this relation by monitoring the numbers on my own. I added the note in my answer. Jun 5 '20 at 8:37