I know about derivatives of easy functions like $\frac{dx^2}{dx}=2x$ and such, but now that I'm dealing with matrices, I am somewhat at a loss.

Given a matrix function $f(\mathbf{X})$

$$f(\mathbf{X}) =log(y' \cdot exp(\mathbf{U X}z))$$

  • where $y'$ is an m-element row vector
  • $\mathbf{U}$ is an $m\times (m-1)$ , static matrix
  • $\mathbf{X}$ is an $n\times (m-1)$ matrix
  • $z$ is an (n+1)-element row vector

I am struggling to find the derivative $\frac{df}{d\mathbf{X}} $ and not sure where I should even start.

I wanted to start with the term inside the exp(). While looking around, I found this document that states on the second page $\frac{d\mathbf{AXB}}{d\mathbf{X}} = \mathbf{B}^T \bigotimes \mathbf{A}$ , which I wanted to apply, but there $\mathbf{B}$ is a matrix and not a vector.

Can someone show me how to tackle such a problem?


1 Answer 1


Define some additional variables and their differentials $$\eqalign{ w &= UXz &\implies dw = U\,dX\,z \cr e &= \exp(w) &\implies de = e\odot dw \cr \beta &= y^Te = y:e &\implies d\beta = y:de \cr f &= \log(\beta) &\implies df = d\beta/\beta \cr }$$ where $(\odot)$ denotes the elementwise/Hadamard product
and (:) denotes the trace/Frobenius product, i.e. $\,A:B={\rm tr}(A^TB)$

Successively substitute into that last differential, until we get an expression in terms of $dX$ $$\eqalign{ df &= \beta^{-1}d\beta \cr &= \beta^{-1}y:de \cr &= \beta^{-1}y:e\odot dw \cr &= \beta^{-1}y\odot e:dw \cr &= \beta^{-1}y\odot e:U\,dX\,z \cr &= \beta^{-1}U^T(y\odot e)z^T:dX \cr &= \beta^{-1}U^TYez^T:dX \cr \cr \frac{\partial f}{\partial X} &= \frac{U^TY\exp(UXz)z^T}{y^T\exp(UXz)} \cr }$$ where $Y={\rm Diag}(y)$ was used to eliminate the Hadamard product.

  • $\begingroup$ Thank you for the detailed answer! $\endgroup$
    – deemel
    Commented Dec 13, 2017 at 23:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .