# Derivative of $\log |AA^T|$ with respect to $A$

What is the derivative of $\log |AA^T|$ with respect to $A$, where $|A|$ denotes the determinant of $A$?

Let $$f : \mathbb{R}^{m \times n} \to \mathbb{R}$$ be defined by

$$f (X) = \log |X X^T|$$

The directional derivative of $$f$$ in the direction of $$V \in \mathbb{R}^{m \times n}$$ is

$$\begin{array}{rl} D_V f (X) &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |(X + h V) (X + h V)^T| - \log |X X^T| \right)\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |X X^T + h V X^T + h X V^T + h^2 V V^T| - \log |X X^T| \right)\end{array}$$

If $$X$$ has full row rank, i.e., if $$\operatorname{rank} (X) = m$$, then $$X X^T$$ is invertible. Hence,

$$\begin{array}{rl} D_V f (X) &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |X X^T + h V X^T + h X V^T + h^2 V V^T| - \log |X X^T| \right)\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \left( \log |X X^T| + \log | I_m + h (X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right)| - \log |X X^T| \right)\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \, \log | I_m + h (X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right)|\\\\ &= \displaystyle\lim_{h \to 0} \frac{1}{h} \, \log ( 1 + h \operatorname{tr} ((X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right)))\\\\ &= \displaystyle\lim_{h \to 0} \operatorname{tr} ((X X^T)^{-1} \left( V X^T + X V^T + h V V^T \right))\\\\ &= \operatorname{tr} ((X X^T)^{-1} \left( V X^T + X V^T \right))\\\\ &= 2 \, \operatorname{tr} (X^T (X X^T)^{-1} V)\end{array}$$

• Your argument works only when $X$ is an invertible square matrix.
– user91684
May 25 '16 at 22:11
• @loupblanc Indeed. I have rewritten my answer. Please let me know if you find any errors or unwarranted assumptions. May 25 '16 at 23:35
• Yes, it is a well-written proof.
– user91684
May 26 '16 at 12:56
• @loupblanc Interestingly, the directional derivative of $f$ in the direction of $X$ at $X$ is $$D_X f (X) = 2 \, \operatorname{tr} (X^T (X X^T)^{-1} X) = 2 \, \operatorname{tr} (X X^T (X X^T)^{-1}) = 2 \, \operatorname{tr} (I_m) = 2 m$$ May 26 '16 at 13:05
• Yes, because $f(X+h X)=\log(\det(XX^T))+\log((1+h)^{2m})=\log(\det(XX^T))+2hm+O(h^2)$
– user91684
May 26 '16 at 13:18

Assume that $A\in M_{n,m}$ where $n\leq m$ and $rank(A)=n$ and let $f(A)=\log(\det(AA^T))$. Then $Df_A:H\rightarrow tr((AA^T)^{-1}(HA^T+AH^T))$ or, if $A$ is a derivable matrix function, then $f'(A)=tr((AA^T)^{-1}(A'A^T+AA'^T))$.

Let $M = AA^T$ and $f = \log \det \left( M \right)$.

We will utilize the following the identities

• Trace and Frobenius product relation $$A:B={\rm tr}(A^TB)$$ or $$A^T:B={\rm tr}(AB)$$
• Cyclic property of Trace/Frobenius product \eqalign{ A:BC &= AC^T:B \cr &= B^TA:C \cr &= {\text etc.} \cr }
• Jacobi's formula (for nonsingular matrix $M$) $$\log \det \left( M \right) = {\rm tr}\log\left( M \right)$$

Now, we obtain the differential first and thereafter we obtain the gradient.

So, \begin{align} df &= d \log \det \left( M \right) \\ &= d \ {\rm tr}\left( \log\left( M \right) \right) \hspace{8mm} \text{note: utilized Jacobi's formula} \\ &= {\rm tr} \left( M^{-1} dM \right) \\ &= M^{-T} \ : \ dM \hspace{8mm} \text{note: utilized trace and Frobenius relation} \\ &= \left(AA^T\right)^{-T} \ : \ d\left(AA^T\right) \\ &= \left(AA^T\right)^{-1} \ : \ d\left(AA^T\right) \\ &= \left(AA^T\right)^{-1} \ : \ \left( dA \ A^T + A \ dA^T\right) \\ &= \left(AA^T\right)^{-1} : 2 \ dA \ A^T \\ &= 2 \left(AA^T\right)^{-1} (A^T)^T : dA \hspace{8mm} \text{note: utilized cyclic property of Frobenius product} \\ &= 2 \left(AA^T\right)^{-1} A : dA \\ \end{align}

So, the derivative of $f = \log \det \left( AA^T \right)$ with respect to $A$ is \begin{align} \frac{\partial}{\partial A} f = \frac{\partial}{\partial A} \log \det \left( AA^T \right) = 2 \left(AA^T\right)^{-1} A .\\ \end{align}

Hint 1:

$\partial \log\det(M)=trace(M^{-1} \partial M)$

Hint 2:

Use the chain rule.

I hope this helps. Please let us know if you couldn't figure it out.

• This solved my problem. Thanks :) May 26 '16 at 10:28