# Are the same expression kronecker Product and vec operator representation when differentiating a matrix by a matrix?

When differentiating a matrix by a matrix it is true that dA(X)/dX is the same thing as dvec(A(X))/dvec(X) acorrording to this paper.

So, I compared kronecker product representation with vec operator representation following case.

$$\mathbf{Y}=(2,3)$$, $$\mathbf{X}=(4,2)$$

\begin{aligned} \frac{\partial \mathbf{Y}}{\partial \mathbf{X}} &= \frac{\partial }{\partial \mathbf{X}} \otimes \mathbf{Y} \\[10pt] &=\left[\begin{array}{cc} \dfrac{\partial }{\partial X_{11}} & \dfrac{\partial }{\partial X_{12}} \\ \dfrac{\partial }{\partial X_{21}} & \dfrac{\partial }{\partial X_{22}} \\ \dfrac{\partial }{\partial X_{31}} & \dfrac{\partial }{\partial X_{32}} \\ \dfrac{\partial }{\partial X_{41}} & \dfrac{\partial }{\partial X_{42}} \end{array}\right] \otimes \mathbf{Y} \\[10pt] &= \left[\begin{array}{ccc|ccc} \dfrac{\partial Y_{11}}{\partial X_{11}} & \dfrac{\partial Y_{12}}{\partial X_{11}} & \dfrac{\partial Y_{13}}{\partial X_{11}} & \dfrac{\partial Y_{11}}{\partial X_{12}} & \dfrac{\partial Y_{12}}{\partial X_{12}} & \dfrac{\partial Y_{13}}{\partial X_{12}} \\ \dfrac{\partial Y_{21}}{\partial X_{11}} & \dfrac{\partial Y_{22}}{\partial X_{11}} & \dfrac{\partial Y_{23}}{\partial X_{11}} & \dfrac{\partial Y_{21}}{\partial X_{12}} & \dfrac{\partial Y_{22}}{\partial X_{12}} & \dfrac{\partial Y_{23}}{\partial X_{12}} \\[10pt] \hline \dfrac{\partial Y_{11}}{\partial X_{21}} & \dfrac{\partial Y_{12}}{\partial X_{21}} & \dfrac{\partial Y_{13}}{\partial X_{21}} & \dfrac{\partial Y_{11}}{\partial X_{22}} & \dfrac{\partial Y_{12}}{\partial X_{22}} & \dfrac{\partial Y_{13}}{\partial X_{22}} \\ \dfrac{\partial Y_{21}}{\partial X_{21}} & \dfrac{\partial Y_{22}}{\partial X_{21}} & \dfrac{\partial Y_{23}}{\partial X_{21}} & \dfrac{\partial Y_{21}}{\partial X_{22}} & \dfrac{\partial Y_{22}}{\partial X_{22}} & \dfrac{\partial Y_{23}}{\partial X_{22}} \\[10pt] \hline \dfrac{\partial Y_{11}}{\partial X_{31}} & \dfrac{\partial Y_{12}}{\partial X_{31}} & \dfrac{\partial Y_{13}}{\partial X_{31}} & \dfrac{\partial Y_{11}}{\partial X_{32}} & \dfrac{\partial Y_{12}}{\partial X_{32}} & \dfrac{\partial Y_{13}}{\partial X_{32}} \\ \dfrac{\partial Y_{21}}{\partial X_{31}} & \dfrac{\partial Y_{22}}{\partial X_{31}} & \dfrac{\partial Y_{23}}{\partial X_{31}} & \dfrac{\partial Y_{21}}{\partial X_{32}} & \dfrac{\partial Y_{22}}{\partial X_{32}} & \dfrac{\partial Y_{23}}{\partial X_{32}} \\[10pt] \hline \dfrac{\partial Y_{11}}{\partial X_{41}} & \dfrac{\partial Y_{12}}{\partial X_{41}} & \dfrac{\partial Y_{13}}{\partial X_{41}} & \dfrac{\partial Y_{11}}{\partial X_{42}} & \dfrac{\partial Y_{12}}{\partial X_{42}} & \dfrac{\partial Y_{13}}{\partial X_{42}} \\ \dfrac{\partial Y_{21}}{\partial X_{41}} & \dfrac{\partial Y_{22}}{\partial X_{41}} & \dfrac{\partial Y_{23}}{\partial X_{41}} & \dfrac{\partial Y_{21}}{\partial X_{42}} & \dfrac{\partial Y_{22}}{\partial X_{42}} & \dfrac{\partial Y_{23}}{\partial X_{42}} \end{array} \right] \end{aligned}

and

$$\text{vec}(\mathbf{Y})= \mathbf{y}= \begin{bmatrix} y_{11} & y_{21} & y_{12} & y_{22} & y_{13} & y_{23} \end{bmatrix}^T$$

$$\text{vec}(\mathbf{X})= \mathbf{x}= \begin{bmatrix} x_{11} & x_{21} & x_{31} & x_{41} & x_{12} & x_{22} & x_{32} & x_{42} \end{bmatrix}^T$$

\begin{aligned} \frac{\partial \text{vec}(\mathbf{Y})}{\partial \text{vec}(\mathbf{X})} &= \frac{\partial \mathbf{y}}{\partial \mathbf{x}} \\ &= \left[\begin{array}{cccccc} \dfrac{\partial Y_{11}}{\partial X_{11}} & \dfrac{\partial Y_{21}}{\partial X_{11}} & \dfrac{\partial Y_{12}}{\partial X_{11}} & \dfrac{\partial Y_{22}}{\partial X_{11}} & \dfrac{\partial Y_{13}}{\partial X_{11}} & \dfrac{\partial Y_{23}}{\partial X_{11}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{21}} & \dfrac{\partial Y_{21}}{\partial X_{21}} & \dfrac{\partial Y_{12}}{\partial X_{21}} & \dfrac{\partial Y_{22}}{\partial X_{21}} & \dfrac{\partial Y_{13}}{\partial X_{21}} & \dfrac{\partial Y_{23}}{\partial X_{21}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{31}} & \dfrac{\partial Y_{21}}{\partial X_{31}} & \dfrac{\partial Y_{12}}{\partial X_{31}} & \dfrac{\partial Y_{22}}{\partial X_{31}} & \dfrac{\partial Y_{13}}{\partial X_{31}} & \dfrac{\partial Y_{23}}{\partial X_{31}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{41}} & \dfrac{\partial Y_{21}}{\partial X_{41}} & \dfrac{\partial Y_{12}}{\partial X_{41}} & \dfrac{\partial Y_{22}}{\partial X_{41}} & \dfrac{\partial Y_{13}}{\partial X_{41}} & \dfrac{\partial Y_{23}}{\partial X_{41}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{12}} & \dfrac{\partial Y_{21}}{\partial X_{12}} & \dfrac{\partial Y_{12}}{\partial X_{12}} & \dfrac{\partial Y_{22}}{\partial X_{12}} & \dfrac{\partial Y_{13}}{\partial X_{12}} & \dfrac{\partial Y_{23}}{\partial X_{12}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{22}} & \dfrac{\partial Y_{21}}{\partial X_{22}} & \dfrac{\partial Y_{12}}{\partial X_{22}} & \dfrac{\partial Y_{22}}{\partial X_{22}} & \dfrac{\partial Y_{13}}{\partial X_{22}} & \dfrac{\partial Y_{23}}{\partial X_{22}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{32}} & \dfrac{\partial Y_{21}}{\partial X_{32}} & \dfrac{\partial Y_{12}}{\partial X_{32}} & \dfrac{\partial Y_{22}}{\partial X_{32}} & \dfrac{\partial Y_{13}}{\partial X_{32}} & \dfrac{\partial Y_{23}}{\partial X_{32}} \\[5pt] \dfrac{\partial Y_{11}}{\partial X_{42}} & \dfrac{\partial Y_{21}}{\partial X_{42}} & \dfrac{\partial Y_{12}}{\partial X_{42}} & \dfrac{\partial Y_{22}}{\partial X_{42}} & \dfrac{\partial Y_{13}}{\partial X_{42}} & \dfrac{\partial Y_{23}}{\partial X_{42}} \end{array} \right] \end{aligned}

If these results are the same, how do I transpose $$\frac{\partial }{\partial \mathbf{X}} \otimes \mathbf{Y}$$ to make it equal to $$\frac{\partial \text{vec}(\mathbf{Y})}{\partial \text{vec}(\mathbf{X})}$$?

The derivative operation is a distraction, so let's omit it.

Given the matrices \eqalign{ X &\in {\mathbb R}^{m\times n} \cr Y &\in {\mathbb R}^{p\times q} \cr } and their Kronecker products \eqalign{ A &= X\otimes Y &\in {\mathbb R}^{mp\times nq} \cr B &= {\rm vec}(X)\otimes {\rm vec}(Y)^T &\in {\mathbb R}^{mn\times pq}\cr } you want to know how to transform $$A \implies B$$

The elements in these matrices are identical. The only difference is in the matrix dimensions and in the ordering of the elements.

Vectorizing the matrices yields two vectors related by a permutation. \eqalign{ P\,{\rm vec}(A) &= {\rm vec}(B) \cr Pa &= b \cr P_{ij}\,a_j &= b_i \cr } The elements of the permuation are easy to find:
$$P_{ij}$$ equals one if the $$j^{th}$$ element of $$a$$ maps to the $$i^{th}$$ element of $$b$$.

The single-entry matrix $$E_{mn}$$ has all elements equal to zero, except for the $$(m,n)$$ element which is equal to one. Replacing $$(X,Y)$$ by single-entry matrices leads to $$(A,B)$$ becoming single-entry matrices and $$(a,b)$$ becoming single-entry vectors.

The non-zero elements of $$P$$ can be found by iterating through every possible single-entry matrix for the input matrices $$X \,{\rm and}\, Y$$. Then setting $$\,P_{ij}=1$$, where the non-zero elements of the output vectors $$(b,a)$$ are observed at $$(b_i,a_j)$$.

The permutation depends only on the dimensions of the starting matrices. For your derivative expressions the dimensions are $$(m,n,p,q)=(4,2,2,3)$$ and the transformation is
$${\rm vec}\Bigg(\frac{\partial{\rm vec}(Y)}{\partial{\rm vec}(X)}\Bigg) = P\,\,{\rm vec}\Bigg(\frac{\partial}{\partial X}\otimes Y\Bigg)$$