-3
$\begingroup$

I have trouble in calculating partial derivatives of matrix recently. The problem is described as following \begin{align} \frac{\partial \log \det\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)}{\partial \mathbf{Q}} \end{align} where $\mathbf{I}$ is identity matrix. $\mathbf{\Sigma}$ and $\mathbf{Q}$ are square matrix. $\det(\cdot)$ is determinant operation. I wanna compute this formula. thanks a lot.

$\endgroup$
  • $\begingroup$ Hint: use the fact that the derivative of $\log \det X$ with respect to $X$ is $X^{-1}$ and apply the chain rule. $\endgroup$ – Brian Borchers Sep 4 '18 at 2:29
  • $\begingroup$ @Qiuyun Kindly show some of the things you may have tried or your thoughts about the problem. As it stands, the question would be considered lacking in effort by the community and is likely to be downvoted and closed. $\endgroup$ – The Long Night Sep 4 '18 at 4:17
  • $\begingroup$ @BrianBorchers Thank you for your answer! I have solved this formula and the result is given by next answer. $\endgroup$ – Qiuyun.Zou Sep 4 '18 at 9:34
  • $\begingroup$ @ DevashishKaushik Thank you for your answer. I will keep it in mind and show more information I have thought. $\endgroup$ – Qiuyun.Zou Sep 4 '18 at 9:39
1
$\begingroup$

Define the matrix $$X=(I+\Sigma Q)$$ Write the function in terms of this new matrix, then find the differential and gradient. $$\eqalign{ \lambda &= \log\det X \cr d\lambda &= X^{-T}:dX = X^{-T}:\Sigma\,dQ = \Sigma^TX^{-T}:dQ \cr \frac{\partial\lambda}{\partial Q} &= \Sigma^TX^{-T} \cr }$$ Since $(\Sigma, Q)$ are symmetric, the gradient can be simplified to $$\frac{\partial\lambda}{\partial Q} = \Sigma\,(I+Q\Sigma)^{-1}$$

$\endgroup$
1
$\begingroup$

Solution: \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}= \left[\begin{array}{ccc} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{11}}&\cdots &\frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{1,u}}\\ \vdots & \ddots &\vdots\\ \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{u,1}} &\cdots & \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{u,u}} \end{array} \right] \end{align} So the key is to find $\frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}}$ for $(i,j)\in [u,u]$.

To this end,we apply the fact \begin{align} \frac{\partial \log \det (\mathbf{A})}{\partial x}=\text{tr}\left({\mathbf{A}^{-1}\frac{\partial \mathbf{A}}{\partial x}}\right) \end{align} where $\mathbf{A}$ is function of $x$, i.e., $\mathbf{A}=\mathbf{A}(x)$. (The proof of above formula can be given by $\frac{\partial \log \det(A)}{\partial x}=\frac{1}{\det(\mathbf{A})}\frac{\partial \det(\mathbf{A})}{\partial x}$ and Jacobi's formula).

With this knowledge, we have \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}} &\overset{(a)}{=}\text{tr} \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}\boldsymbol{e}_i\boldsymbol{e}_j^T}\right)\\ &\overset{(b)}{=}\boldsymbol{e}_j^T \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}}\right) \boldsymbol{e}_i \end{align} where $\boldsymbol{e}_i$ is a vector with all zero elements expect i-th element being 1 in step $(a)$, and $(b)$ holds via the property of trace operation.

Here, $\mathbf{\Sigma}$ and $\mathbf{Q}$ are symmetric matrix. We then have \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}} =\boldsymbol{e}_i^T \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}}\right) \boldsymbol{e}_j \end{align} Thus, we can get \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}=(\mathbf{I}+\mathbf{\Sigma Q})^{-1}\mathbf{\Sigma} \end{align}

By the way, if $\mathbf{\Sigma}$ and $\mathbf{Q}$ are not symmetric, then the result is given by \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}=\mathbf{\Sigma}^T((\mathbf{I}+\mathbf{\Sigma Q})^{-1})^T \end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.