hadamard product differentiation How to differentiate the following formula with respect to $\bf Y$
$\bf b^T (Y \odot Y)c$
in which the $\bf b$ and $\bf c$ are vectors. $\bf Y$ is a non-square matrix. and $\odot$ denotes hadamard product.
 A: Write the function in terms of the Frobenius Inner Product, then find its differential and gradient
$$\eqalign{
 f &= bc^T:Y\circ Y \cr\cr 
df &= bc^T: d\,(Y\circ Y) \cr 
   &= bc^T: (2\,Y\circ dY) \cr 
   &= (2\,Y\circ bc^T): dY \cr\cr 
\frac{\partial f}{\partial Y} &= 2\,Y\circ bc^T \cr\cr 
}$$
Now that you have the gradient, you want to press on and find the hessian.  The problem is that the gradient is a matrix, therefore the hessian will be a 4th order tensor! 
One way to proceed is vectorization
$$\eqalign{
 G &= (2bc^T)\circ Y \cr
 g &= {\rm vec}(G) \cr
   &= {\rm vec}(2bc^T)\circ y \cr
   &= {\rm Diag}({\rm vec}(2bc^T))\,y \cr\cr
dg &= {\rm Diag}({\rm vec}(2bc^T))\,dy \cr\cr
\frac{\partial g}{\partial y} &= {\rm Diag}({\rm vec}(2bc^T)) \cr\cr
}$$
Another way to proceed is via Einstein-index notation. To make things easier we'll use a special 3rd-order tensor whose components $T_{ijk}=1$ when $i\!=\!j\!=\!k$ but are zero otherwise.  Note that this tensor is symmetric wrt all of its indices.
$$\eqalign{
 G_{jn} &= 2 b_i c_m\,\, T_{ijk} T_{mnp}\,\, Y_{kp} \cr\cr
\frac{\partial G_{jn}}{\partial Y_{rs}} &= 2 b_i c_m\,\, T_{ijk} T_{mnp}\,\, \delta_{kr} \delta_{ps} \cr
  &= 2 b_i c_m\,\, T_{ijr} T_{mns} \cr
}$$
