1
$\begingroup$

I have a covariance matrix defined as a rank-one matrix plus a diagonal matrix, with free parameters including a scalar $k$ and a column vector $\vec{v}$. This covariance matrix can be written as $\Sigma=kvv^T+(1-k)I\circ{vv^T}$. I am interested in the derivative of the log-determinant of this covariance matrix $\Sigma$ with respect to each of the element in $\vec{v}$ and with respect to $k$.

Checking on some online materials, I found the derivation and formula that $\frac{\partial\ln|A|}{\partial{x}}=Tr(A^{-1}\frac{\partial{A}}{\partial{x}})$. However, I still get stuck at deriving $\frac{\partial\Sigma}{\partial v}$ and $\frac{\partial\Sigma}{\partial{k}}$.

A related question, if I am interested in the derivative of $\Sigma$ with respect to each of the element in vector $\vec{v}$, should I still think of it as a question in which one is interested in the derivative of a matrix with respect to a vector $\frac{\partial\Sigma}{\partial v}$?

$\endgroup$

1 Answer 1

3
$\begingroup$

Define the variables $$\eqalign{ F &= {\large\tt 1} - I \cr P &= I+kF \cr S &= \Sigma = P\circ vv^T \cr }$$ Write the function of interest in terms of these variables and find its differential. $$\eqalign{ \phi &= \log(\det(S)) \cr d\phi &= S^{-1}:dS \cr }$$ Expand $dS$ in terms of $dv$ and find the gradient wrt $v$. $$\eqalign{ d\phi &= S^{-1}:P\circ(dv\,v^T+v\,dv^T) \cr &= 2P\circ S^{-1}:dv\,v^T \cr &= 2(P\circ S^{-1})v:dv \cr \frac{\partial \phi}{\partial v} &= 2(P\circ S^{-1})v \cr }$$ Expand $dS$ in terms of $dk$ and find the gradient wrt $k$. $$\eqalign{ d\phi &= S^{-1}:dP\circ vv^T \cr &= S^{-1}:(dk\,F)\circ vv^T \cr &= S^{-1}:({\large\tt 1}-I)\circ vv^T\,dk \cr &= S^{-1}:(vv^T-I\circ vv^T)\,dk \cr \frac{\partial\phi}{\partial k} &= S^{-1}:(vv^T-I\circ vv^T) \cr\cr }$$ In the steps above, a colon denotes the trace/Frobenius product, i.e. $$A:B = {\rm Tr}(A^TB)$$ which commutes with the elementwise/Hadamard product $$A:B\circ C = A\circ B:C$$ and, of course, the Hadamard and Frobenius products are themselves commutative $$\eqalign{ A:B &= B:A \cr A\circ B &= B\circ A \cr }$$

$\endgroup$
3
  • $\begingroup$ I collected all of the terms involving $k$ into the definition of $F$. Now I've made an edit to keep them as separate variables. Hopefully that makes it easier to follow. $\endgroup$
    – greg
    Jul 8, 2019 at 14:07
  • 1
    $\begingroup$ Still reading the derivation. Just at the first few equations, it seems to me that $P$ should be equal to $I+kF$, no? $\endgroup$
    – cml
    Jul 8, 2019 at 15:29
  • $\begingroup$ Indeed $P=I+kF\;$ Thanks to all the commenters. The answer has been updated. $\endgroup$
    – greg
    Jul 8, 2019 at 16:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .