Thank you for taking the time to read my question. I've had a look at other posts regarding differentiation of Hadamard Products of matrices but none of the examples are the same as the one I am curious about. I am looking to do the following:

$ \frac{dL}{d\beta} $ where $L = y'(X\beta \: ⊙ \: X\beta ) $ where y is a N by 1, X is N by K and $\beta$ is K by 1. Denote $\underline{x_i'}$ the i-th row of the matrix X.

Here are my thoughts : $y'(X\beta \: ⊙ \: X\beta ) = y_1(\underline{x_1'}\beta)^{2}+...+y_n(\underline{x_n'}\beta)^{2}$ so its derivative for any entry $\beta_k$ in $\beta$ will be equal to $2y_1 x_{1,k}\underline{x_1'}\beta \: + \: ... +2 y_n x_{n,k}\underline{x_n'}\beta = 2<y⊙\underline{x_k}, X\beta >$ which suggests that, in matrix form, it should be something like $2y'X ⊙ X\beta$ ? This is only my intuition.

Many thanks for any help and time you may provide.

  • $\begingroup$ Can you please clarify what is $y'$? Your idea seems to be generally in the right ballpark, but details are uncertain. I would think when differentiating a square there should be a factor of $2$... $\endgroup$ – Zach Teitler Dec 26 '17 at 2:41
  • $\begingroup$ Thank you. $y'$ is the transpose of the vector $y$. $\endgroup$ – Louis Dec 26 '17 at 10:08
  • $\begingroup$ Do you mean $2y'(X \odot X)\beta$? I just want to make sure how you are wanting to associate the matrix products. $\endgroup$ – Zach Teitler Dec 27 '17 at 23:29

The trace/Frobenius product is $$A:B={\rm tr}(A^TB)$$ The Frobenius and Hadamard products commute with each other and themselves $$\eqalign{ A\odot B &= B\odot A \cr A:B &= B:A \cr (A\odot B):C &= A:(B\odot C) \cr }$$ which allows us to write $$\eqalign{ L &= y:(X\beta\odot X\beta) \cr dL &= y:(X\,d\beta\odot X\beta+X\beta\odot X\,d\beta) \cr &= y:2(X\beta\odot X\,d\beta) \cr &= 2(X\beta\odot y):X\,d\beta \cr &= 2X^T(X\beta\odot y):d\beta \cr \frac{\partial L}{\partial\beta} &= 2X^T(X\beta\odot y) \cr }$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.