How to compute derivative with Hadamard product? Let $\mathbf{x}$, $\mathbf{y}$ and $\mathbf{z}$ are $n$-dimensional column vector, and
$$f =  \mathbf{x}\circ \mathbf{y} \circ\mathbf{z}$$
Here $\circ$ is the element-wise Hadamard product. Then how to compute the gradient
$\frac{\partial f }{\partial\mathbf{x}}$, $\frac{\partial f }{\partial\mathbf{y}}$, and $\frac{\partial f }{\partial\mathbf{z}}$?
My solution is 
$$\frac{\partial f }{\partial\mathbf{x}} = \mathbf{y} \circ\mathbf{z}, \frac{\partial f }{\partial\mathbf{y}} = \mathbf{x} \circ\mathbf{z},\frac{\partial f }{\partial\mathbf{z}} = \mathbf{x} \circ\mathbf{y}$$
Is it correct?
thanks.
 A: You can replace the Hadamard product of a vector by a matrix product with the diagonal matrix formed from that vector, e.g. let
$$\eqalign{
X &= {\rm Diag}(x), \quad
Y &= {\rm Diag}(y), \quad
Z &= {\rm Diag}(z) \\
}$$
then the function can be written in a number of ways
$$\eqalign{
f &= x\circ y\circ z \\&= YZx = ZXy = XYz \\
}$$
The differential of the function is 
$$\eqalign{
df &= YZ\,dx + ZX\,dy + XY\,dz \\
}$$
yielding the gradients 
$$\eqalign{
\frac{\partial f}{\partial x} &= YZ, \quad
\frac{\partial f}{\partial y} &= ZX, \quad
\frac{\partial f}{\partial z} &= XY \\
}$$
Since diagonal matrices commute, the order of multiplication can be reversed.
A: If $f\colon \Bbb R^n\times \Bbb R^n \times \Bbb R^n \to \Bbb R^n$ is given by $f({\bf x}, {\bf y}, {\bf z}) = {\bf x}\circ {\bf y} \circ {\bf z}$, then $f$ is trilinear. Thus its total derivative is the linear map ${\rm D}f({\bf x},{\bf y},{\bf z}): \Bbb R^n \times \Bbb R^n \times \Bbb R^n \to \Bbb R^n$ given by $${\rm D}f({\bf x},{\bf y}, {\bf z})({\bf h}_1,{\bf h}_2, {\bf h}_3) = {\bf h}_1 \circ {\bf y} \circ {\bf z} + {\bf x}\circ {\bf h}_2 \circ {\bf z} + {\bf x}\circ {\bf y} \circ {\bf h}_3. $$So we get that $$\frac{\partial f}{\partial {\bf x}}({\bf x},{\bf y},{\bf z}){\bf h} \equiv {\rm D}_{\bf x}f({\bf x},
{\bf y},{\bf z})({\bf h}) = {\bf h}\circ {\bf y} \circ {\bf z},$$ and similarly for the other partial derivatives. Since $\circ$ is commutative, you can write $$\frac{\partial f}{\partial {\bf x}}({\bf x},{\bf y},{\bf z}) = {\bf y}\circ {\bf z}\circ \_.$$
