# Differentiation of the Hadamard Matrix Product

Thank you for taking the time to read my question. I've had a look at other posts regarding differentiation of Hadamard Products of matrices but none of the examples are the same as the one I am curious about. I am looking to do the following:

$\frac{dL}{d\beta}$ where $L = y'(X\beta \: ⊙ \: X\beta )$ where y is a N by 1, X is N by K and $\beta$ is K by 1. Denote $\underline{x_i'}$ the i-th row of the matrix X.

Here are my thoughts : $y'(X\beta \: ⊙ \: X\beta ) = y_1(\underline{x_1'}\beta)^{2}+...+y_n(\underline{x_n'}\beta)^{2}$ so its derivative for any entry $\beta_k$ in $\beta$ will be equal to $2y_1 x_{1,k}\underline{x_1'}\beta \: + \: ... +2 y_n x_{n,k}\underline{x_n'}\beta = 2<y⊙\underline{x_k}, X\beta >$ which suggests that, in matrix form, it should be something like $2y'X ⊙ X\beta$ ? This is only my intuition.

Many thanks for any help and time you may provide.

• Can you please clarify what is $y'$? Your idea seems to be generally in the right ballpark, but details are uncertain. I would think when differentiating a square there should be a factor of $2$... – Zach Teitler Dec 26 '17 at 2:41
• Thank you. $y'$ is the transpose of the vector $y$. – Louis Dec 26 '17 at 10:08
• Do you mean $2y'(X \odot X)\beta$? I just want to make sure how you are wanting to associate the matrix products. – Zach Teitler Dec 27 '17 at 23:29

The trace/Frobenius product is $$A:B={\rm tr}(A^TB)$$ The Frobenius and Hadamard products commute with each other and themselves \eqalign{ A\odot B &= B\odot A \cr A:B &= B:A \cr (A\odot B):C &= A:(B\odot C) \cr } which allows us to write \eqalign{ L &= y:(X\beta\odot X\beta) \cr dL &= y:(X\,d\beta\odot X\beta+X\beta\odot X\,d\beta) \cr &= y:2(X\beta\odot X\,d\beta) \cr &= 2(X\beta\odot y):X\,d\beta \cr &= 2X^T(X\beta\odot y):d\beta \cr \frac{\partial L}{\partial\beta} &= 2X^T(X\beta\odot y) \cr }