5
$\begingroup$

What is the derivative of $\log |AA^T|$ with respect to $A$, where $|A|$ denotes the determinant of $A$?

$\endgroup$
0
3
$\begingroup$

Let $f : \mathbb{R}^{m \times n} \to \mathbb{R}$ be defined by

$$f (X) = \log |X X^T|$$

The directional derivative of $f$ in the direction of $V \in \mathbb{R}^{m \times n}$ is

$$\begin{array}{rl} D_V f (X) &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |(X + h V) (X + h V)^T| - \log |X X^T| \right)\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |X X^T + h V X^T + h X V^T + h^2 V V^T| - \log |X X^T| \right)\end{array}$$

If $X$ has full row rank, i.e., if $\operatorname{rank} (X) = m$, then $X X^T$ is invertible. Hence,

$$\begin{array}{rl} D_V f (X) &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |X X^T + h V X^T + h X V^T + h^2 V V^T| - \log |X X^T| \right)\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |X X^T| + \log | I_m + h (X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right)| - \log |X X^T| \right)\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \, \log | I_m + h (X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right)|\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \, \log ( 1 + h \operatorname{tr} ((X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right)))\\\\ &= \displaystyle\lim_{h \to 0} \operatorname{tr} ((X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right))\\\\ &= \operatorname{tr} ((X X^T)^{-1} \left( V X^T + X V^T \right))\\\\ &= 2 \, \operatorname{tr} (X^T (X X^T)^{-1} V)\end{array}$$


$\endgroup$
5
  • 1
    $\begingroup$ Your argument works only when $X$ is an invertible square matrix. $\endgroup$
    – user91684
    May 25 '16 at 22:11
  • $\begingroup$ @loupblanc Indeed. I have rewritten my answer. Please let me know if you find any errors or unwarranted assumptions. $\endgroup$ May 25 '16 at 23:35
  • 1
    $\begingroup$ Yes, it is a well-written proof. $\endgroup$
    – user91684
    May 26 '16 at 12:56
  • $\begingroup$ @loupblanc Interestingly, the directional derivative of $f$ in the direction of $X$ at $X$ is $$D_X f (X) = 2 \, \operatorname{tr} (X^T (X X^T)^{-1} X) = 2 \, \operatorname{tr} (X X^T (X X^T)^{-1}) = 2 \, \operatorname{tr} (I_m) = 2 m$$ $\endgroup$ May 26 '16 at 13:05
  • 1
    $\begingroup$ Yes, because $f(X+h X)=\log(\det(XX^T))+\log((1+h)^{2m})=\log(\det(XX^T))+2hm+O(h^2)$ $\endgroup$
    – user91684
    May 26 '16 at 13:18
3
$\begingroup$

Assume that $A\in M_{n,m}$ where $n\leq m$ and $rank(A)=n$ and let $f(A)=\log(\det(AA^T))$. Then $Df_A:H\rightarrow tr((AA^T)^{-1}(HA^T+AH^T))$ or, if $A$ is a derivable matrix function, then $f'(A)=tr((AA^T)^{-1}(A'A^T+AA'^T))$.

$\endgroup$
2
$\begingroup$

Let $M = AA^T$ and $f = \log \det \left( M \right)$.

We will utilize the following the identities

  • Trace and Frobenius product relation $$A:B={\rm tr}(A^TB)$$ or $$A^T:B={\rm tr}(AB)$$
  • Cyclic property of Trace/Frobenius product $$\eqalign{ A:BC &= AC^T:B \cr &= B^TA:C \cr &= {\text etc.} \cr }$$
  • Jacobi's formula (for nonsingular matrix $M$) $$\log \det \left( M \right) = {\rm tr}\log\left( M \right)$$

Now, we obtain the differential first and thereafter we obtain the gradient.

So, \begin{align} df &= d \log \det \left( M \right) \\ &= d \ {\rm tr}\left( \log\left( M \right) \right) \hspace{8mm} \text{note: utilized Jacobi's formula} \\ &= {\rm tr} \left( M^{-1} dM \right) \\ &= M^{-T} \ : \ dM \hspace{8mm} \text{note: utilized trace and Frobenius relation} \\ &= \left(AA^T\right)^{-T} \ : \ d\left(AA^T\right) \\ &= \left(AA^T\right)^{-1} \ : \ d\left(AA^T\right) \\ &= \left(AA^T\right)^{-1} \ : \ \left( dA \ A^T + A \ dA^T\right) \\ &= \left(AA^T\right)^{-1} : 2 \ dA \ A^T \\ &= 2 \left(AA^T\right)^{-1} (A^T)^T : dA \hspace{8mm} \text{note: utilized cyclic property of Frobenius product} \\ &= 2 \left(AA^T\right)^{-1} A : dA \\ \end{align}

So, the derivative of $f = \log \det \left( AA^T \right)$ with respect to $A$ is \begin{align} \frac{\partial}{\partial A} f = \frac{\partial}{\partial A} \log \det \left( AA^T \right) = 2 \left(AA^T\right)^{-1} A .\\ \end{align}

$\endgroup$
1
$\begingroup$

Hint 1:

$\partial \log\det(M)=trace(M^{-1} \partial M)$

Hint 2:

Use the chain rule.

I hope this helps. Please let us know if you couldn't figure it out.

$\endgroup$
1
  • $\begingroup$ This solved my problem. Thanks :) $\endgroup$ May 26 '16 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.