Differentiating Kronecker product of a vector with respect to itself I am trying to differentiate the Kronecker product of a vector with respect to itself:
$$ \frac{\partial}{\partial \pmb{\delta}} (\pmb{\delta} \otimes \pmb{\delta} ) $$
That is part of this larger differentiation:
 $$
\begin{equation}
\label{eq1}
\tag{1}
\frac{\partial}{\partial \pmb{\delta}} \left(\pmb{\delta}^\intercal \mathbf{A} (\pmb{\delta} \otimes \pmb{\delta} ) \right)
\end{equation}
$$
Where:
-$\pmb{\delta}_{K \times 1}$ is a vector and $\pmb{\delta}^{\intercal}_{1\times K}$ is its transpose.
-$ \mathbf{A}_{K \times K^2}$ is constant matrix that is independent from $\pmb{\delta}$ ($ \partial \mathbf{A}/ \partial \pmb{\delta}=0) $.
Using definitions that can be found in Matrix Calculus textbooks, I have simplified it as follows:
$$
\begin{split}
 \frac{\partial}{\partial \pmb{\delta}} (\pmb{\delta} \otimes \pmb{\delta} ) 
&=\frac{\partial \pmb{\delta}}{\partial \pmb{\delta}} \otimes \pmb{\delta}+ [\mathbf{I}_K \otimes \mathbf{U_{K^2 \times K^2}}][\frac{\partial \pmb{\delta}}{\partial \pmb{\delta}} \otimes \pmb{\delta}][\mathbf{I}_1 \otimes \mathbf{I}_{1\times 1} ]\\
&=vec(\mathbf{I}_K) \otimes \pmb{\delta}+[\mathbf{I}_K \otimes \mathbf{U_{K^2 \times K^2}}][vec(\mathbf{I}_K) \otimes \pmb{\delta}]\\
&=[\mathbf{I}_{K^3}+\mathbf{I}_K \otimes \mathbf{U_{K^2 \times K^2}}][vec(\mathbf{I}_K) \otimes \pmb{\delta}]
\end{split}
$$
Where:


*

*$\mathbf{I}_n$ is $n \times n$ unit matrix

*$vec(\mathbf{B})$ is the classic vectorization operator.

*$\mathbf{U}_{r \times s}$ is the permutation matrix as it is defined in classic matrix calculus textbooks (e.g. here is shown with $T_{m,n}$).  


Can the above equation get simplified further? My goal is to collect whatever form of $ \pmb{\delta}$ that appears as a result of equation \ref{eq1} to the right hand side of the term. To be more precise the furthest I got from \ref{eq1} is here:
$$
\begin{equation}
\label{eq2}
\tag{2}
\begin{split}
\frac{\partial}{\partial \pmb{\delta}} \left(\pmb{\delta}^\intercal \mathbf{A} (\pmb{\delta} \otimes \pmb{\delta} ) \right)&
=\mathbf{A} (\pmb{\delta} \otimes \pmb{\delta} ) \\
&+
[\mathbf{I}_K \otimes (\pmb{\delta}^\intercal \mathbf{A})][\mathbf{I}_{K^3}+\mathbf{I}_K \otimes \mathbf{U_{K^2 \times K^2}}][vec(\mathbf{I}_K) \otimes \pmb{\delta}]
\end{split}
\end{equation}
$$
and I am trying to collect the $\pmb{\delta}$ in the second term of  the right hand side of above equation on the right hand side like the first term.
 A: For ease of typing, let's use $\,x={\pmb\delta}\,$ as the independent variable.
We'll also use the fact that the Kronecker product of two vectors can be expanded in two ways:
$$a\otimes b = (I_a\otimes b)\,a = (a\otimes I_b)\,b$$ where $I_a$ is the identity matrix whose dimensions are compatible with the $a$ vector, while $I_b$ is compatible with the $b$ vector.
The differential and Jacobian of first function can be calculated as
$$\eqalign{
  y &= x\otimes x \cr
 dy &= dx\otimes x + x\otimes dx = \big(I\otimes x + x\otimes I\big)\,dx \cr
J = \frac{\partial y}{\partial x} &= I\otimes x + x\otimes I \cr
}$$
Using the previous result $\big(dy=J\,dx\big),\,$ your second function can be easily dispatched 
$$\eqalign{
\phi &= x^TAy = y^TA^Tx \cr
d\phi
 &= x^TA\,dy + y^TA^T\,dx  \cr
 &= \big(x^TAJ + y^TA^T\big)\,dx \cr
 &= \big(Ay + J^TA^Tx\big)^T\,dx \cr
\frac{\partial\phi}{\partial x}
 &= Ay + J^TA^Tx \cr
 &= A(x\otimes x) + \big(I\otimes x^T + x^T\otimes I\big)A^Tx \cr
}$$
The first term is the same between you, Frank, and me. 
My second term appears to be the transpose of Frank's and different from yours.
A: Consider a 3rd order tensor ${\mathcal K}$ which allows you to express the Kronecker product of two vectors $a\in{\mathbb R}^m$, $b\in{\mathbb R}^p$ using dot products 
$$\eqalign{
 c = a\otimes b &= a\cdot{\mathcal K}\cdot b \cr
}$$
where all of the tensor components ${\mathcal K}_{i\alpha k}$ are equal zero or one; the nonzero components occurring when  $\big\{\,\alpha=p*(i-1)+k\,\big\}$ is satisfied. 
Applying it to the first problem
$$\eqalign{
 f &= x\otimes x \cr\cr
df &= x\otimes dx + dx\otimes x \cr
  &= (I+U)\,(x\otimes dx) \cr
  &= (I+U)\,(x\cdot{\mathcal K}\cdot dx) \cr\cr
\frac{\partial f}{\partial x} &= (I+U)\,(x\cdot{\mathcal K}) \cr
}$$ where $U$ is the permutation (aka Commutation) matrix mentioned in the question.
Applying it to the second problem
$$\eqalign{
 h &= x^TAf \cr
   &= x\cdot A(x\otimes x) \cr
   &= x\cdot A(x\cdot{\mathcal K}\cdot x) \cr\cr
dh &= A(x\cdot{\mathcal K}\cdot x)\cdot dx + x\cdot A\,df \cr
   &= \Big(A(x\cdot{\mathcal K}\cdot x) + x\cdot A\,(I+U)\,(x\cdot{\mathcal K)}\Big)\cdot dx \cr
\cr
\frac{\partial h}{\partial x}
 &= A(x\cdot{\mathcal K}\cdot x) + (x\cdot A)\,(I+U)\,(x\cdot{\mathcal K)} \cr
 &= A(x\otimes x) + (x^TA)\,\frac{\partial f}{\partial x} \cr\cr
}$$
Notice that the final result does not depend on ${\mathcal K}$ or dot products, so you should be able to derive it via another route that does not utilize these techniques.
A: For convenience, let the independent variable be $x$, which is easier to type than $\delta$. 
Let's use a colon to denote the trace/Frobenius product, i.e.
$$A:B={\rm tr}(A^TB)$$
Start by finding the SVD of the constant matrix
$$A=(US)V^T = WV^T = \sum_k w_kv_k^T = w_kv_k^T$$
The last expression uses an index summation convention where repeated indices imply summation.
Let's work on the objective function 
$$\eqalign{
f &= x:w_kv_k^T(x\otimes x) \cr
  &= (w_k:x)\,(v_k:(x\otimes x)) \cr
}$$
Now we need to factor the vectors $$v_k=\sum_j b_{kj}\otimes c_{kj}$$ such that each factor has the same dimensions as $x$. This factorization is a bit unusual, but search for papers by vanLoan and Pitsianis, or Pitsianis' 1997 dissertation.
With these factorizations, we can write
$$\eqalign{
f &= (w_k:x)\,(b_{kj}\otimes c_{kj}):(x\otimes x)) \cr
  &= (w_k:x)(b_{kj}:x)(c_{kj}:x) \cr
}$$
We've managed to factor the function into the product of 3 scalars, a form which is very easy to differentiate.
$$\eqalign{
df &= \Big[(b_{kj}:x)(c_{kj}:x)w_k + (w_k:x)(c_{kj}:x)b_{kj} + (w_k:x)(b_{kj}:x)c_{kj}\Big]:dx 
\cr\cr
\frac{\partial f}{\partial x}
 &= (b_{kj}:x)(c_{kj}:x)w_k + (w_k:x)(c_{kj}:x)b_{kj} + (w_k:x)(b_{kj}:x)c_{kj} \cr 
 &= \sum_k \sum_j \,\Big(x^Tb_{kj}\,x^Tc_{kj}\,w_k + x^Tw_k\,x^Tc_{kj}\,b_{kj} + x^Tw_k\,x^Tb_{kj}\,c_{kj}\Big) \cr 
}$$
