Derivative of log-likelihood cost function with respect to a matrix Recently, I am learning derivative method to a function and thanks to @hans help, I can solve those which can be expressed by Frobenius product. But for the log-likelihood function, I do not how to deal with. Here is the example functions:
$$L = -\sum_{i,j}ln(1 + exp(-(2X_{ij} - 1)(U_{ik}V_{kj})))$$
where, $X \in \mathbb{R}^{m \times n}$, $U \in \mathbb{R}^{m \times k}$ and $V \in \mathbb{R}^{k \times n}$.
I would like to get the $\frac{\partial{L}}{\partial{W}}$, where $V$ is fixed as constant? Can you give me some hints how to get the answer?
Thanks.
 A: Define a few variables for convenience
$$\eqalign{
 W &= UV \cr
 A &= (1-2X)\circ W \cr &= W - 2X\circ W \cr
 E &= \exp(A) \cr
 Q &= \log(1+E) \cr
}$$
where log() is applied element-wise.
Now write your function and find its differential
$$\eqalign{
 L &= -1:Q \cr\cr
dL &= -1:\frac{1}{1+E}\circ dE \cr
   &= \frac{-1}{1+E}: E\circ dA \cr
   &= \frac{-E}{1+E}: (dW - 2dX\circ W - 2X\circ dW) \cr
   &= \frac{E\circ(2X-1)}{1+E}:dW - \frac{2E\circ W}{1+E}:dX \cr
   &= \frac{E\circ(2X-1)}{1+E}:(U\,dV+dU\,V) - \frac{2E\circ W}{1+E}:dX \cr
   &= U^T\Big(\frac{E\circ(2X-1)}{1+E}\Big):dV + \Big(\frac{E\circ(2X-1)}{1+E}\Big)V^T:dU - \frac{2E\circ W}{1+E}:dX \cr
}$$
where I've used the notation $\frac{1}{Y}$ to represent the element-wise (Hadamard) inverse of the matrix $Y$.
To find the gradient with respect to any of these variables, set the differentials of the other 2 to zero.  
For example, to find the gradient wrt X set $(dV,dU)$ to zero yielding
$$\eqalign{
 \frac{\partial L}{\partial X} &= \frac{-2E\circ W}{1+E} \cr
}$$
