# Kronecker product and outer product confusion

I have two column vectors:

$$u = \left[\matrix{ 1 \cr 2\cr }\right]$$

$$v = \left[\matrix{ 4 \cr 4\cr }\right]$$

I'm trying to compute the Kronecker product of two vectors $u \otimes v$.

As I understand, the outer product of vectors is a special case of the Kronecker product of matrices.

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product $A \otimes\ B$ is the mp × nq block matrix.

So will $u \otimes v$ be of dimension 4 × 1 (according to the first definition) or 2 × 2 (according to the second definition)?

• The product $u \otimes v$ should be 4-by-1 if you follow the Wikipedia article on the Kronecker product strictly, but it should be 2-by-2 if you follow the Wikipedia article on the outer product strictly. This inconsistency is nothing serious. Both matrices have the same entries; they are just arranged differently. You should pick one that suits the way you're going to use the result. – Tunococ May 16 '14 at 4:30

This is a very good example of abuse of notation, more precisely, reload of operator. Actually the operator $\otimes$ is usually used as tensor product, which is a bilinear operator. It's easy to verify that both Kronecker product (denoted by $\otimes_K$) and outer product (denoted by $\otimes_O$) are bilinear and special forms of tensor product. For example, given two vectors $u,v\in V$, we have $$u\otimes_O v=u\otimes_Kv^H$$ This is why wiki says outer product is a special case of Kronecter product.

The relation between outer product $$\circ$$ and kronecker product $$\otimes$$ . $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} = vec(({\textbf{a}}\circ {\textbf{b}})^T) = vec(({\textbf{a}}{\textbf{b}^T}_{I \times J})^T_{J \times I})_{JI \times 1}$$

Before taking vectorization you should do a transpose.

In case of 3 vectors, the resulted matrix as before is multiplied by each entry in c vector and each result is a frontal slice of a tensor. Notice that $$.*$$ is an element wise product between each element in $$\textbf{c}$$ vector and the resultant matrix of $$(\textbf{a} \circ \textbf{b})$$ and not the element wise product between the whole vector $$\textbf{c}$$ and $$(\textbf{a} \circ \textbf{b})$$. $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} \otimes {\textbf{c}}_{K \times 1} = vec(( ({\textbf{a}}\circ {\textbf{b}}) .* {\textbf{c}})^T_{I \times J \times K})_{IJK \times 1}$$

Outer product between 2, each of 2 dimension then the result would be 4 dimension tensor which is totally different than the kronecker product

Kronecker Product for matrices $${\textbf{A}}_{I \times R} \otimes {\textbf{B}}_{ J \times R} = {\textbf{D}}_{ IJ \times R^2}$$

$${\textbf{D}} = \begin{bmatrix} \vdots & \vdots & & \vdots & & \vdots \\ \textbf{a}_1 \otimes {\textbf{b}_1} & {\textbf{a}_1} \otimes {\textbf{b}_2} & \dots & \textbf{a}_1 \otimes {\textbf{b}_R} & \dots &\textbf{a}_R \otimes {\textbf{b}_R} \\ \vdots & \vdots & & \vdots & & \vdots \end{bmatrix}$$

Outer Product for matrices $$\textbf{A}_{I \times R_1} \circ \textbf{B}_{J \times R_2} = {\textbf{C}}_{I \times J \times R_1 \times R_2}$$ to show how the result would be you have to do it column by column outer product, $${\textbf{a}_1} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{y}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{a}_1} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{n}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{p} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{j} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix},$$

The final result would be, $${\textbf{C}}_{I \times J \times 1 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{p}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{y} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 1 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{n} & {\textbf{j}} \\ \vdots & \vdots \end{bmatrix}$$

• What is the symbol $.*$ that you use? Elementwise product? If that is the case, the dimensions does not seem to add upp. Are you thinking about broadcasting like in some programming language? – LudvigH May 27 at 13:47
• it is not element wise product with the vector c, instead it is element wise product with each element in that vector, each element in c is multiplied by the whole matrix of (aob) and this is one slice as the frontal slice for k=1 and at the end the result is a tensor of IxJxK size as c vector is of size K. Actually, I was working on my master thesis when I answered this question and currently, I am learning OOP programming creating User interfaces, I hope that I can utilize what I have learned here and there. – Mour_Ka Jun 5 at 8:31
• Could you please consider describing this in the answer, or adding a reference to some literature, so that the answer is complete? – LudvigH Jun 5 at 8:35
• I actually found no literature to describe this and this is how I found the question, I actually found this relation by monitoring the numbers on my own. I added the note in my answer. – Mour_Ka Jun 5 at 8:37