15
$\begingroup$

I have two column vectors:

\begin{equation} u = \left[\matrix{ 1 \cr 2\cr }\right] \end{equation}

\begin{equation} v = \left[\matrix{ 4 \cr 4\cr }\right] \end{equation}

I'm trying to compute the Kronecker product of two vectors $u \otimes v$.

As I understand, the outer product of vectors is a special case of the Kronecker product of matrices.

http://en.wikipedia.org/wiki/Kronecker_product says:

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product $A \otimes\ B$ is the mp × nq block matrix.

enter image description here

http://en.wikipedia.org/wiki/Outer_product says:

enter image description here

So will $u \otimes v$ be of dimension 4 × 1 (according to the first definition) or 2 × 2 (according to the second definition)?

$\endgroup$
2
  • 1
    $\begingroup$ The product $u \otimes v$ should be 4-by-1 if you follow the Wikipedia article on the Kronecker product strictly, but it should be 2-by-2 if you follow the Wikipedia article on the outer product strictly. This inconsistency is nothing serious. Both matrices have the same entries; they are just arranged differently. You should pick one that suits the way you're going to use the result. $\endgroup$
    – Tunococ
    May 16, 2014 at 4:30
  • $\begingroup$ See also mathworld.wolfram.com/KroneckerProduct.html and mathworld.wolfram.com/VectorDirectProduct.html. $\endgroup$
    – user688486
    Dec 1, 2021 at 11:49

2 Answers 2

16
$\begingroup$

This is a very good example of abuse of notation, more precisely, reload of operator. Actually the operator $\otimes$ is usually used as tensor product, which is a bilinear operator. It's easy to verify that both Kronecker product (denoted by $\otimes_K$) and outer product (denoted by $\otimes_O$) are bilinear and special forms of tensor product. For example, given two vectors $u,v\in V$, we have $$u\otimes_O v=u\otimes_Kv^H$$ This is why wiki says outer product is a special case of Kronecter product.

$\endgroup$
2
  • $\begingroup$ Thanks, (I'd upvote if I could), I guess this explains the slide 7 of ima.umn.edu/industrial/2006-2007/kolda/kolda.pdf that says Observe: For two vectors a and b, $a\otimes_O b$ and $a\otimes_K b$ have the same elements, but one is shaped into a matrix and the other into a vector. $\endgroup$
    – remind
    May 16, 2014 at 15:45
  • $\begingroup$ another related question: Kruskal Tensor: sum of outer or Kronecker products?. If you have some time to check it out that would be very helpful. Thanks! $\endgroup$
    – remind
    May 16, 2014 at 17:14
6
$\begingroup$

The relation between outer product $\circ$ and kronecker product $\otimes$ . $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} = vec(({\textbf{a}}\circ {\textbf{b}})^T) = vec(({\textbf{a}}{\textbf{b}^T}_{I \times J})^T_{J \times I})_{JI \times 1}$$

Before taking vectorization you should do a transpose.

In case of 3 vectors, the resulted matrix as before is multiplied by each entry in c vector and each result is a frontal slice of a tensor. Notice that $.*$ is an element wise product between each element in $\textbf{c}$ vector and the resultant matrix of $(\textbf{a} \circ \textbf{b})$ and not the element wise product between the whole vector $\textbf{c}$ and $(\textbf{a} \circ \textbf{b})$. $$ {\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} \otimes {\textbf{c}}_{K \times 1} = vec(( ({\textbf{a}}\circ {\textbf{b}}) .* {\textbf{c}})^T_{I \times J \times K})_{IJK \times 1} $$

Outer product between 2, each of 2 dimension then the result would be 4 dimension tensor which is totally different than the kronecker product

Kronecker Product for matrices $$ {\textbf{A}}_{I \times R} \otimes {\textbf{B}}_{ J \times R} = {\textbf{D}}_{ IJ \times R^2} $$

$$ {\textbf{D}} = \begin{bmatrix} \vdots & \vdots & & \vdots & & \vdots \\ \textbf{a}_1 \otimes {\textbf{b}_1} & {\textbf{a}_1} \otimes {\textbf{b}_2} & \dots & \textbf{a}_1 \otimes {\textbf{b}_R} & \dots &\textbf{a}_R \otimes {\textbf{b}_R} \\ \vdots & \vdots & & \vdots & & \vdots \end{bmatrix}$$

Outer Product for matrices $$ \textbf{A}_{I \times R_1} \circ \textbf{B}_{J \times R_2} = {\textbf{C}}_{I \times J \times R_1 \times R_2}$$ to show how the result would be you have to do it column by column outer product, $$ {\textbf{a}_1} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{y}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{a}_1} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{n}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{p} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{j} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix},$$

The final result would be, $${\textbf{C}}_{I \times J \times 1 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{p}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{y} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 1 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{n} & {\textbf{j}} \\ \vdots & \vdots \end{bmatrix}$$

$\endgroup$
6
  • 1
    $\begingroup$ What is the symbol $.*$ that you use? Elementwise product? If that is the case, the dimensions does not seem to add upp. Are you thinking about broadcasting like in some programming language? $\endgroup$
    – LudvigH
    May 27, 2020 at 13:47
  • $\begingroup$ it is not element wise product with the vector c, instead it is element wise product with each element in that vector, each element in c is multiplied by the whole matrix of (aob) and this is one slice as the frontal slice for k=1 and at the end the result is a tensor of IxJxK size as c vector is of size K. Actually, I was working on my master thesis when I answered this question and currently, I am learning OOP programming creating User interfaces, I hope that I can utilize what I have learned here and there. $\endgroup$
    – Mour_Ka
    Jun 5, 2020 at 8:31
  • 1
    $\begingroup$ Could you please consider describing this in the answer, or adding a reference to some literature, so that the answer is complete? $\endgroup$
    – LudvigH
    Jun 5, 2020 at 8:35
  • $\begingroup$ I actually found no literature to describe this and this is how I found the question, I actually found this relation by monitoring the numbers on my own. I added the note in my answer. $\endgroup$
    – Mour_Ka
    Jun 5, 2020 at 8:37
  • 2
    $\begingroup$ The symbol $\circ$ is utilized for Hadamard product: en.wikipedia.org/wiki/Hadamard_product_(matrices) $\endgroup$ Aug 3, 2022 at 19:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .