I am seeking to compute the following:

$\nabla ( \mathbf{1}^T f(X) )$


$\mathbf{1}^T = (1 , \ldots , 1)$ an $1 \times n$ vector.

$f(X)= (A*X) \circ (B*X)$

$A$ and $B$: $n \times n$ matrices

$X$: $n \times 1$ vector

$\circ$ the Hadamard or element wise product

$*$ matrix product.

I read the following rule $d(X \circ Y) = d(X) \circ Y + d(Y) \circ X$

But in the above I have a mixture of matrix and element wise product (dimensions are consistent at the end $\mathbf{1}^T f(X) $ is of size $1 \times 1$)

  • $\begingroup$ Prove that your function equals $x^TA^TBx$ and take it from there. $\endgroup$ – Hans Engler Jun 2 '17 at 15:13
  • $\begingroup$ thank you very much I worked it out! $\endgroup$ – Sha Jun 8 '17 at 14:49

Let's denote the elementwise/Hadamard and inner/Frobenius products respectively as $$\eqalign{ A &= B\circ C \cr \alpha &= B:C = {\rm tr}(B^TC) \cr }$$ Recall that these products are commutative, and mutually commutative $$\eqalign{ A\circ B &= B\circ A \cr A:B &= B:A \cr A:B\circ C &= A\circ B:C \cr }$$ and that the matrix of all ones is the identity element for the Hadamard product. Note that the matrices $(A,B,C)$ must have the same shape for these products to make sense.

Your scalar function can be written as $$\eqalign{ y &= 1:f \cr &= 1:(Ax)\circ(Bx) \cr &= Ax:Bx \cr }$$ Whosee differential and gradient are $$\eqalign{ dy &= A\,dx:Bx + Ax:B\,dx \cr &= Bx:A\,dx + Ax:B\,dx \cr &= (A^TB + B^TA)\,x:dx \cr \cr \frac{\partial y}{\partial x} &= (A^TB + B^TA)\,x \cr\cr }$$ (Note that I've used juxtaposition rather than $*$ for the ordinary matrix product.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.