Derivative of a function w.r.t. matrix $K$ and scalar $\sigma^2$ Suppose that $K$ is an $r\times r$ symmetric and positive definite matrix, and that $D=(\sigma^2+\sigma^2_{\epsilon})^{-1} I$ is a $n\times n$ matrix with scalars $\sigma^2>0$ and $\sigma^2_{\epsilon}>0$. Let $\Sigma= SKS'+D^{-1}$ be an $n\times n$ positive definite matrix with $r<n$. Now we are interested in finding the second derivative of function $f$ w.r.t. unknown parameters $K$ and $\sigma^2$ respectively:
$$ f(K, \sigma^2) = \log |\Sigma| + z' \Sigma^{-1} z,
$$
where $z$ is an $n \times 1$ vector.
Notice that $$|\Sigma|= |K^{-1} + S'DS|\cdot |K| \cdot |D^{-1}|,$$
and $$\Sigma^{-1} = D-DS(K^{-1}+S'DS)^{-1}S' D.$$
And we have $\frac{\partial^2 \log|K|}{\partial K^2} = - K^{-2}$.
Any suggestions will be appreciated!
Thanks,
 A: You can rearrange the function, and write it in terms of the Frobenius (:) product as
$$\eqalign{
 f &= \log(\det(\Sigma)) + z^T\Sigma^{-1}z \cr
   &= {\rm tr}(\log(\Sigma)) + zz^T:\Sigma^{-1} \cr
}$$
Now take the differential of the function and make successive variable substitutions until you get it terms of the variables that you are interested in
$$\eqalign{
 df &= \Sigma^{-T}:d\Sigma + zz^T:d\Sigma^{-1} \cr
    &= \Sigma^{-1}:d\Sigma - zz^T:\Sigma^{-1}\,d\Sigma\,\Sigma^{-1} \cr
    &= \Sigma^{-1}:d\Sigma - \Sigma^{-1}zz^T\Sigma^{-1}:d\Sigma \cr
    &= (\Sigma^{-1} - \Sigma^{-1}zz^T\Sigma^{-1}):d\Sigma \cr
    &= B:d\Sigma \cr
    &= B:(S\,dK\,S^T + dD^{-1}) \cr
    &= B:(S\,dK\,S^T + Id\sigma^2) \cr
    &= S^TBS:dK + B:Id\sigma^2 \cr
    &= S^TBS:dK + {\rm tr}(B)\,d\sigma^2 \cr
}$$
Now setting $dK=0$ yields the derivative with respect to $\sigma^2$
$$\eqalign{
  \frac{\partial f}{\partial\sigma^2} &= {\rm tr}(B) \cr
    &= {\rm tr}(\Sigma^{-1} - \Sigma^{-1}zz^T\Sigma^{-1}) \cr
}$$
And setting $d\sigma^2=0$ yields
$$\eqalign{
  \frac{\partial f}{\partial K} &= S^TBS \cr
    &= S^T(\Sigma^{-1} - \Sigma^{-1}zz^T\Sigma^{-1})S \cr
}$$
