As derivative of $l_{p}$- norm is \begin{align*} \frac{\partial}{\partial\mathbf{x}}{||\mathbf{x}||}_{p} &= \frac{\mathbf{x} |\mathbf{x}|^{p-2}}{{||\mathbf{x}||}_{p}^{p-1}} \end{align*}

I want to find $\nabla log(||H||_{1})$, where $H$ is positive matrix. So, the chain rule is \begin{align*} \nabla log(||H||_{1}) &= log(||H||_{1})' (||H||_{1})'\\ &= \frac{1}{||H||_{1}}\frac{H |H|^{1-2}}{{||H||}_{1}^{1-1}}\\ &= \frac{1}{||H||_{1}}\\ \end{align*}

But the answer seems to be

\begin{align*} \nabla log(||H||_{1}) &= &= \frac{H}{||H||_{1}}\\ \end{align*}

What am I missing here? Where is $H$ at numerator from?

  • $\begingroup$ "Seems to be" answer can't be right even for reasons of homogeneity. The logarithm is essentially 0-degree homogeneous (up to an additive constant) so its derivative is homogeneous of degree $-1$. $\endgroup$
    – user357151
    Jun 27 '18 at 13:26

Apply the sign function element-wise to the matrix $H$ $$S = {\rm sign}(H)$$ Use this to write the Manhattan norm as $$\eqalign{ \|H\|_1 &= S:H \cr }$$ where the colon denotes the trace/Frobenius product, i.e. $\,\,A:B={\rm tr}(A^TB).$

Use this to calculate the logarithmic derivative of the Manhattan norm as
$$\eqalign{ \Omega &= \log(\|H\|_1) \cr d\Omega &= \frac{d\|H\|_1}{\|H\|_1} = \frac{S:dH}{\|H\|_1} \cr \frac{\partial\Omega}{\partial H} &= \frac{S}{\|H\|_1} \cr\cr }$$ If all elements of $H>0\,$ then the numerator is the matrix $S=1$ (as you expected).
In any case, the numerator is definitely not $H$.

  • $\begingroup$ If the log function is $\nabla log(||H||_{1} + \epsilon)$ (where $\epsilon$ is a small positive number), I think the answer does not change? $\endgroup$
    – Jan
    Jun 28 '18 at 4:50
  • 1
    $\begingroup$ @Jan The gradient will change (the "smallness" of $\epsilon$ is irrelevant) becoming $$\frac{\partial\Omega}{\partial H} = \frac{S}{\|H\|_1+\epsilon}$$ $\endgroup$
    – greg
    Jun 28 '18 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.