# Setting

$$f:\mathbb{R}^n\to\mathbb{R}^n$$ and $$\|\cdot \|$$ be the usual Euclidean norm. I would like to compute the derivative with respect to $$x$$ of $$\phi(x) = \frac{f(x)}{\|f(x)\|}$$

# My Attempt at a Solution

$$\nabla_x \frac{f(x)}{||f(x)||} = \frac{\nabla_x f(x)}{||f(x)||} + f(x)\nabla_x\left(f(x)^\top f(x)\right)^{-\frac{1}{2}} = \frac{\nabla_x f(x)}{||f(x)||}-\frac{1}{2}\frac{f(x)}{||f(x)||^3} 2f(x)^\top \nabla_x f(x) = \left(I - \frac{f(x)f(x)^\top}{||f(x)||^2}\right)\frac{\nabla_x f(x)}{||f(x)||}$$ However I am very unsure about this. In particular, I have a feeling that the second term would be $$f(x)^\top \nabla_x f(x)$$ and hence lead to $$\nabla_x \frac{f(x)}{||f(x)||} = \frac{\nabla_x f(x)}{||f(x)||} - \frac{\nabla_x f(x)}{||f(x)||} = 0$$ However this wouldn't make much sense cause surely the gradient is not $$0$$ for every function.

• do you have any assumptions on $f$? Jul 13, 2021 at 11:03
• I am happy to assume what I need for this to work. For instance, happy for $g$ to be Lipschitz continuous and/or bounded. @NickCastillo
– user318854
Jul 13, 2021 at 11:05
• @NickCastillo So in practice $g$ will be the gradient function of another function $f:\mathbb{R}^n\to\mathbb{R}$. I also know that the Hessian function $H:\mathbb{R}^n\to\mathbb{R}^{n\times n}$ is well defined (i.e. the Hessian $H(x) = \nabla_x g(x)$ exists)
– user318854
Jul 13, 2021 at 11:06

$$\def\p{{\partial}} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}}$$Separate $$f$$ into two components: $$\;$$ its length $$(\lambda)$$ and direction $$(\phi)$$ \eqalign{ \lambda^2 &= \|f\|^2 &\quad\implies\quad \lambda\,d\lambda = \c{f^Tdf} \\ \phi &= \lambda^{-1} f \\ } Calculate the differential of $$\phi$$ \eqalign{ d\phi &= \lambda^{-1} df - \lambda^{-2}f\,{d\lambda} \\ &= \lambda^{-1} df - \lambda^{-3}f\,{(\lambda\,d\lambda)} \\ &= \lambda^{-1}I\, df - \lambda^{-3}f\,\c{(f^Tdf)} \\ &= \lambda^{-1}\left(I - \phi\phi^T\right)df \\ } Now differentiate with respect to $$x$$ \eqalign{ \grad{\phi}{x} &= \left(\frac{I - \phi\phi^T}{\lambda}\right)\grad{f}{x} \\ } So your initial solution was absolutely correct. And you can prove the famous result about unit vectors being perpendicular to their gradients \eqalign{ \phi^T\left(\grad{\phi}{x}\right) &= \left(\frac{\phi^T-({\tt1})\phi^T}{\lambda}\right)\grad{f}{x} \\ &= \left(\frac{0}{\lambda}\right)\grad{f}{x} \\ &= 0 \\ }

• This is a great solution. One thing that is bothering me. How $\lambda d\lambda=f^T df$? In particular, why "$f^T$". Jul 14, 2021 at 8:35
• @user550103, you have $\lambda^2=\lambda*\lambda=\|f\|^2=f^Tf$, so $d(\lambda*\lambda) = (d\lambda)\lambda + \lambda(d\lambda)= 2\lambda d\lambda$, the same for $df^Tf=(df^T)f + f^Tdf=2f^Tdf$, back to the equality: $2\lambda d\lambda = 2f^Tdf \implies \lambda d\lambda = f^Tdf$ Jul 14, 2021 at 10:00
• @MathLearner Thanks. Indeed (this is what happens when you want to go on vacation soon...) Jul 14, 2021 at 13:47

As the bets for gradient are low, let's compute the total derivative of $$\phi$$ as follows: \begin{align} d_p\phi(v)&= \frac{d_pf(v)\|f(p)\|-f(p)\frac{1}{2\|f(p)\|}\cdot2\langle d_pf(v),f(p\rangle}{\|f(p)\|^2}\\ &=\frac{1}{\|f(p)\|}\left(d_pf(v)-\frac{f(p)}{\|f(p)\|}\langle \frac{f(p)}{\|f(p)\|},d_pf(v)\rangle, \right), \end{align} which is quite a pleasant result as it is the perpendicular of $$d_pf(v)$$ to $$f(p)$$, scaled by the length of $$f(p)$$.

I will present the calculation in standard coordinates. Your $$\phi$$ is the composition of $$u(x)=x/\|x\|$$ and $$f(x)$$, i.e. $$\phi_i(x) = u_i(f(x)), \quad u_i(x) = \frac{x_i}{\|x\|}.$$ For $$x$$ away from the origin, as $$u_i(x) = -\partial_i {\|x\|}$$, and since $$(1/x)' = -1/{x^2}$$, product and chain rule gives $$\partial_k u_i = \frac{\delta_{ik}}{\|x\|} - \frac{x_i x_k}{\|x\|^3}.$$ Or, if you prefer, $$\nabla u = \frac1{\|x\|}I_n - \frac1{\|x\|^3}xx^T$$. So long as $$f$$ avoids the origin, chain rule gives $$\partial_j \phi_i = \sum_{k=1}^n\partial_k u_i(f(x))\partial_j f_k(x) = \frac{\partial_jf_i(x)}{\|f(x)\|} - \frac1{\|f(x)\|^3}\sum_{k=1}^n f_i(x) f_k(x) \partial_j f_k$$ i.e. the Jacobian is $$\nabla \phi = \frac{\nabla f}{\|f\|} - \frac1{\|f\|^3}ff^T\nabla f.$$

Regarding your final comment on the feeling: the error in your intuition is that you replaced $$ff^T\in \mathbb R^{n\times n}$$ with the scalar $$f^T f = \|f\|^2$$. This is in fact (as you basically said) what makes the two terms different. In fact, this is already apparent from the above formula for $$\nabla u = \partial_i\partial_j \|x\|$$.

As $$\|x\|$$ "scales" like one power of $$x$$, two derivatives should "scale" like $$x^{1-2}=1/x$$. Which is precisely what we see, except in the case of dimension $$n=1$$ where the derivative is indeed exactly $$0$$ whereever defined (!!) In higher dimensions, there are simply more functions that have the same scaling, leading to your error.