I know about derivatives of easy functions like $\frac{dx^2}{dx}=2x$ and such, but now that I'm dealing with matrices, I am somewhat at a loss.

Given a matrix function $f(\mathbf{X})$

$$f(\mathbf{X}) =log(y' \cdot exp(\mathbf{U X}z))$$

  • where $y'$ is an m-element row vector
  • $\mathbf{U}$ is an $m\times (m-1)$ , static matrix
  • $\mathbf{X}$ is an $n\times (m-1)$ matrix
  • $z$ is an (n+1)-element row vector

I am struggling to find the derivative $\frac{df}{d\mathbf{X}} $ and not sure where I should even start.

I wanted to start with the term inside the exp(). While looking around, I found this document that states on the second page $\frac{d\mathbf{AXB}}{d\mathbf{X}} = \mathbf{B}^T \bigotimes \mathbf{A}$ , which I wanted to apply, but there $\mathbf{B}$ is a matrix and not a vector.

Can someone show me how to tackle such a problem?


Define some additional variables and their differentials $$\eqalign{ w &= UXz &\implies dw = U\,dX\,z \cr e &= \exp(w) &\implies de = e\odot dw \cr \beta &= y^Te = y:e &\implies d\beta = y:de \cr f &= \log(\beta) &\implies df = d\beta/\beta \cr }$$ where $(\odot)$ denotes the elementwise/Hadamard product
and (:) denotes the trace/Frobenius product, i.e. $\,A:B={\rm tr}(A^TB)$

Successively substitute into that last differential, until we get an expression in terms of $dX$ $$\eqalign{ df &= \beta^{-1}d\beta \cr &= \beta^{-1}y:de \cr &= \beta^{-1}y:e\odot dw \cr &= \beta^{-1}y\odot e:dw \cr &= \beta^{-1}y\odot e:U\,dX\,z \cr &= \beta^{-1}U^T(y\odot e)z^T:dX \cr &= \beta^{-1}U^TYez^T:dX \cr \cr \frac{\partial f}{\partial X} &= \frac{U^TY\exp(UXz)z^T}{y^T\exp(UXz)} \cr }$$ where $Y={\rm Diag}(y)$ was used to eliminate the Hadamard product.

| cite | improve this answer | |
  • $\begingroup$ Thank you for the detailed answer! $\endgroup$ – deemel Dec 13 '17 at 23:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.