I have two column vectors:

\begin{equation} u = \left[\matrix{ 1 \cr 2\cr }\right] \end{equation}

\begin{equation} v = \left[\matrix{ 4 \cr 4\cr }\right] \end{equation}

I'm trying to compute the Kronecker product of two vectors $u \otimes v$.

As I understand, the outer product of vectors is a special case of the Kronecker product of matrices.

http://en.wikipedia.org/wiki/Kronecker_product says:

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product $A \otimes\ B$ is the mp × nq block matrix.

enter image description here

http://en.wikipedia.org/wiki/Outer_product says:

enter image description here

So will $u \otimes v$ be of dimension 4 × 1 (according to the first definition) or 2 × 2 (according to the second definition)?

  • 1
    $\begingroup$ The product $u \otimes v$ should be 4-by-1 if you follow the Wikipedia article on the Kronecker product strictly, but it should be 2-by-2 if you follow the Wikipedia article on the outer product strictly. This inconsistency is nothing serious. Both matrices have the same entries; they are just arranged differently. You should pick one that suits the way you're going to use the result. $\endgroup$
    – Tunococ
    May 16 '14 at 4:30

This is a very good example of abuse of notation, more precisely, reload of operator. Actually the operator $\otimes$ is usually used as tensor product, which is a bilinear operator. It's easy to verify that both Kronecker product (denoted by $\otimes_K$) and outer product (denoted by $\otimes_O$) are bilinear and special forms of tensor product. For example, given two vectors $u,v\in V$, we have $$u\otimes_O v=u\otimes_Kv^H$$ This is why wiki says outer product is a special case of Kronecter product.


The relation between outer product $\circ$ and kronecker product $\otimes$ . $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} = vec(({\textbf{a}}\circ {\textbf{b}})^T) = vec(({\textbf{a}}{\textbf{b}^T}_{I \times J})^T_{J \times I})_{JI \times 1}$$

Before taking vectorization you should do a transpose.

In case of 3 vectors, the resulted matrix as before is multiplied by each entry in c vector and each result is a frontal slice of a tensor. Notice that $.*$ is an element wise product between each element in $\textbf{c}$ vector and the resultant matrix of $(\textbf{a} \circ \textbf{b})$ and not the element wise product between the whole vector $\textbf{c}$ and $(\textbf{a} \circ \textbf{b})$. $$ {\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} \otimes {\textbf{c}}_{K \times 1} = vec(( ({\textbf{a}}\circ {\textbf{b}}) .* {\textbf{c}})^T_{I \times J \times K})_{IJK \times 1} $$

Outer product between 2, each of 2 dimension then the result would be 4 dimension tensor which is totally different than the kronecker product

Kronecker Product for matrices $$ {\textbf{A}}_{I \times R} \otimes {\textbf{B}}_{ J \times R} = {\textbf{D}}_{ IJ \times R^2} $$

$$ {\textbf{D}} = \begin{bmatrix} \vdots & \vdots & & \vdots & & \vdots \\ \textbf{a}_1 \otimes {\textbf{b}_1} & {\textbf{a}_1} \otimes {\textbf{b}_2} & \dots & \textbf{a}_1 \otimes {\textbf{b}_R} & \dots &\textbf{a}_R \otimes {\textbf{b}_R} \\ \vdots & \vdots & & \vdots & & \vdots \end{bmatrix}$$

Outer Product for matrices $$ \textbf{A}_{I \times R_1} \circ \textbf{B}_{J \times R_2} = {\textbf{C}}_{I \times J \times R_1 \times R_2}$$ to show how the result would be you have to do it column by column outer product, $$ {\textbf{a}_1} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{y}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{a}_1} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{n}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{p} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{j} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix},$$

The final result would be, $${\textbf{C}}_{I \times J \times 1 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{p}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{y} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 1 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{n} & {\textbf{j}} \\ \vdots & \vdots \end{bmatrix}$$

  • $\begingroup$ What is the symbol $.*$ that you use? Elementwise product? If that is the case, the dimensions does not seem to add upp. Are you thinking about broadcasting like in some programming language? $\endgroup$
    – LudvigH
    May 27 '20 at 13:47
  • $\begingroup$ it is not element wise product with the vector c, instead it is element wise product with each element in that vector, each element in c is multiplied by the whole matrix of (aob) and this is one slice as the frontal slice for k=1 and at the end the result is a tensor of IxJxK size as c vector is of size K. Actually, I was working on my master thesis when I answered this question and currently, I am learning OOP programming creating User interfaces, I hope that I can utilize what I have learned here and there. $\endgroup$
    – Mour_Ka
    Jun 5 '20 at 8:31
  • $\begingroup$ Could you please consider describing this in the answer, or adding a reference to some literature, so that the answer is complete? $\endgroup$
    – LudvigH
    Jun 5 '20 at 8:35
  • $\begingroup$ I actually found no literature to describe this and this is how I found the question, I actually found this relation by monitoring the numbers on my own. I added the note in my answer. $\endgroup$
    – Mour_Ka
    Jun 5 '20 at 8:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.