Product rule for Hadamard product differentation? Is there a "simple" solution to
$\bf \frac{\partial}{\partial w}\big(w \odot f(w)\big)$
assuming the matrix $\bf \frac{\partial f}{\partial w}$ is known? With simple I mean something like in the normal vector multiplication case 
$\bf \frac{\partial}{\partial w}\big(w^Tf(w)\big) = f(w) + \big[\frac{\partial f(w)}{\partial w}\big]^T w$
such that no other knowledge of $\bf f(w)$ is required.  
 A: Let Diag denote the function which creates a diagonal matrix from a vector. 
Define some variables for convenience 
$$\eqalign{
 F &= \operatorname{Diag}(f) \cr
 W &= \operatorname{Diag}(w) \cr
 h &= w\odot f = f\odot w \cr
 J &= \frac{\partial f}{\partial w} \cr
\cr
}$$
Now find the differential and gradient of $h$
$$\eqalign{
 dh &= f\odot dw + w\odot df \cr
    &= F\,dw + W\,(J\,dw) \cr
\cr
\frac{\partial h}{\partial w} &= F + WJ \cr
}$$
A: Given $\mathrm f : \mathbb R^n \to \mathbb R^n$, let vector field $\mathrm g : \mathbb R^n \to \mathbb R^n$ be defined by
$$\mathrm g (\mathrm w) : = \begin{bmatrix} w_1 \, f_1 (\mathrm w)\\ w_2 \, f_2 (\mathrm w)\\ \vdots\\ w_n \, f_n (\mathrm w)\end{bmatrix}$$
Differentiating $g_i (\mathrm w)$ with respect to $w_j$,
$$\frac{\partial g_i}{\partial w_j} = \delta_{ij} \, f_i + w_i \, \frac{\partial f_i}{\partial w_j}$$
Hence, the Jacobian of $\mathrm g$ is
$$\frac{\partial \mathrm g}{\partial \mathrm w} = \mbox{diag} (\mathrm f (\mathrm w)) + \mbox{diag} (\mathrm w) \, \frac{\partial \mathrm f}{\partial \mathrm w}$$
