Issues in calculating the gradient I am trying to calculate the gradient of a certain expression. I am not sure if it's possible. I have the following
$f(\alpha_1,\alpha_2,\Lambda) = \log(|2Q_1+2Q_2 +2Q_3|)$
$Q_1$ is a diagonal matrix with the diagonal terms equal to $\alpha_1$
$Q_2$ is a diagonal matrix with the diagonal terms equal to $\alpha_2$
$Q_3$ is a matrix with a bunch of parameters.
Now how can I take the gradient of the function $f$ wrt $\alpha_1$, $\alpha_2$ and the params lets say $\Lambda_{ii}$. Is it possible to have something
 A: Well, if you are interested in finding gradient with the respect to the parameters ($\alpha_1, \alpha_2, \Lambda_{ii}$) separately (without concatenating them in one vector and searching the derivative wrt a vector) you can use some matrix calculus identities (but first you can pull out the factor of $2$ out of the logarithm):
$$\frac{\partial \ln|\mathbf{U}|}{\partial x} ={\rm tr}\left(\mathbf{U}^{-1}\frac{\partial \mathbf{U}}{\partial x}\right)$$
Here $x$ is $\alpha_1, \alpha_2$ or $\Lambda_{ii}$ and matrix $U$ is $\mathbf{Q_1+Q_2+Q_3}$.
So
$$
\begin{eqnarray}
\frac{\partial f(\alpha_1,\alpha_2,\Lambda)}{\partial \alpha_1}={\rm tr}\left((\mathbf{Q_1+Q_2+Q_3})^{-1}\frac{\partial \mathbf{\mathbf{Q_1}}}{\partial \alpha_1}\right)\\
\frac{\partial f(\alpha_1,\alpha_2,\Lambda)}{\partial \alpha_2}={\rm tr}\left((\mathbf{Q_1+Q_2+Q_3})^{-1}\frac{\partial \mathbf{\mathbf{Q_2}}}{\partial \alpha_2}\right) \\
\frac{\partial f(\alpha_1,\alpha_2,\Lambda)}{\partial \Lambda_{ii}}={\rm tr}\left((\mathbf{Q_1+Q_2+Q_3})^{-1}\frac{\partial \mathbf{\mathbf{Q_3}}}{\partial \Lambda_{ii}}\right)
\end{eqnarray}
$$
