I learned from The Matrix Cookbook that the gradient of the $\log \det$ function is given by

\begin{equation} \nabla \log \text{det}(\mathbf{X}^\top \mathbf{X})=2\mathbf{X}(\mathbf{X}^\top \mathbf{X})^{-1}, \end{equation}

where $\mathbf{X}\in\mathbb{R}^{n\times r}$. I wonder which function will give the gradient

\begin{equation} 2\mathbf{A} \mathbf{A}^\top \mathbf{X}(\mathbf{X}^\top \mathbf{X})^{-1}, \end{equation}

for some matrix $\mathbf{A}\in \mathbb{R}^{n\times r}$.

  • $\begingroup$ The gradient of a function is a vector not a matrix. Your first equation does not make sense for me. $\endgroup$ – Dog_69 Aug 4 '18 at 21:29
  • 2
    $\begingroup$ Hi, I'm arranging it in a matrix form. It is similar to the fact that the gradient of $\frac{1}{2}\|\mathbf{X}\|_{\mathrm{F}}^2$ is $\mathbf{X}$. $\endgroup$ – Wuchen Aug 4 '18 at 22:18
  • $\begingroup$ In this case the $r$ of $\mathbb R^{n\times r}$ should be $1$1. $\endgroup$ – Dog_69 Aug 5 '18 at 5:04
  • $\begingroup$ Where does this constraint come from? $\endgroup$ – Wuchen Aug 6 '18 at 14:49
  • 3
    $\begingroup$ @Dog_69 See math.stackexchange.com/questions/2807864/… $\endgroup$ – Jean-Claude Arbaut Aug 7 '18 at 6:03

In general, there is no solution. That follows is a counter-example for the existence of $g$ s.t. $\nabla(g)=2BX(X^TX)^{-1}$ for every $X$ s.t. $rank(X)=r$ (where $B\in M_n$).

Let $n=2,r=1, B=\begin{pmatrix}a&b\\c&d\end{pmatrix},X=[x,y]^T\not= [0,0]^T$.

$\dfrac{\partial g}{\partial x}=\dfrac{2}{x^2+y^2}(ax+by),\dfrac{\partial g}{\partial y}=\dfrac{2}{x^2+y^2}(cx+dy)$. Thus

$\dfrac{\partial^2 g}{\partial x \partial y}=\dfrac{2}{(x^2+y^2)^2}(b(x^2-y^2)-2axy),\dfrac{\partial^2 g}{\partial y \partial x}=\dfrac{2}{(x^2+y^2)^2}(c(y^2-x^2)-2dxy)$.

Thus $g$ exists iff $a=d,b=-c$, that is $B=\begin{pmatrix}a&b\\-b&a\end{pmatrix}$; in particular, $B$ has never the form $AA^T$ except when $B=0$.

EDIT. The general solution of the above equation -with the $(r,\theta)$ polar coordinates- is: $g=2a\log(r)-2b\theta+$ constant.

| cite | improve this answer | |
  • $\begingroup$ @Wuchen , did you read my answer ? $\endgroup$ – user91684 Sep 11 '18 at 22:58
  • $\begingroup$ Thank you! Sorry for the late acceptance. $\endgroup$ – Wuchen Oct 23 '18 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.