Derivative of a function divided by its norm, i.e., $\phi(x) = \frac{f(x)}{\|f(x)\|}$ Setting
$f:\mathbb{R}^n\to\mathbb{R}^n$ and $\|\cdot \|$ be the usual Euclidean norm. I would like to compute the derivative with respect to $x$ of
$$
\phi(x)  = \frac{f(x)}{\|f(x)\|}
$$
My Attempt at a Solution
$$
\nabla_x \frac{f(x)}{||f(x)||} = \frac{\nabla_x f(x)}{||f(x)||} + f(x)\nabla_x\left(f(x)^\top f(x)\right)^{-\frac{1}{2}} = \frac{\nabla_x f(x)}{||f(x)||}-\frac{1}{2}\frac{f(x)}{||f(x)||^3} 2f(x)^\top \nabla_x f(x) = \left(I - \frac{f(x)f(x)^\top}{||f(x)||^2}\right)\frac{\nabla_x f(x)}{||f(x)||}
$$
However I am very unsure about this. In particular, I have a feeling that the second term would be $f(x)^\top \nabla_x f(x)$ and hence lead to
$$
\nabla_x \frac{f(x)}{||f(x)||}  = \frac{\nabla_x f(x)}{||f(x)||} - \frac{\nabla_x f(x)}{||f(x)||} = 0
$$
However this wouldn't make much sense cause surely the gradient is not $0$ for every function.
 A: As the bets for gradient are low, let's compute the total derivative of $\phi$ as follows:
$$
\begin{align}
d_p\phi(v)&=
\frac{d_pf(v)\|f(p)\|-f(p)\frac{1}{2\|f(p)\|}\cdot2\langle d_pf(v),f(p\rangle}{\|f(p)\|^2}\\
&=\frac{1}{\|f(p)\|}\left(d_pf(v)-\frac{f(p)}{\|f(p)\|}\langle \frac{f(p)}{\|f(p)\|},d_pf(v)\rangle,
\right),
\end{align}$$
which is quite a pleasant result as it is the perpendicular of $d_pf(v)$ to $f(p)$, scaled by the length of $f(p)$.
A: I will present the calculation in standard coordinates. Your $\phi$ is the composition of $u(x)=x/\|x\|$ and $f(x)$, i.e.
$$ \phi_i(x) = u_i(f(x)), \quad u_i(x) = \frac{x_i}{\|x\|}.$$   For $x$ away from the origin, as $u_i(x) = -\partial_i {\|x\|}$, and since $(1/x)' = -1/{x^2}$, product and chain rule gives
$$ \partial_k u_i = \frac{\delta_{ik}}{\|x\|} - \frac{x_i x_k}{\|x\|^3}.$$
Or, if you prefer,  $\nabla u = \frac1{\|x\|}I_n - \frac1{\|x\|^3}xx^T$.
So long as  $f$ avoids the origin, chain rule gives
$$ \partial_j \phi_i = \sum_{k=1}^n\partial_k u_i(f(x))\partial_j f_k(x) = \frac{\partial_jf_i(x)}{\|f(x)\|} - \frac1{\|f(x)\|^3}\sum_{k=1}^n f_i(x) f_k(x) \partial_j f_k  $$
i.e. the Jacobian is
$$ \nabla \phi = \frac{\nabla f}{\|f\|} - \frac1{\|f\|^3}ff^T\nabla f.$$

Regarding your final comment on the feeling: the error in your intuition is that you replaced $ff^T\in \mathbb R^{n\times n}$ with the scalar $f^T f = \|f\|^2$. This is in fact (as you basically said) what makes the two terms different. In fact, this is already apparent from the above formula for $\nabla u = \partial_i\partial_j \|x\|$.
As $\|x\|$ "scales" like one power of $x$, two derivatives should "scale" like $x^{1-2}=1/x$. Which is precisely what we see, except in the case of dimension $n=1$ where the derivative is indeed exactly $0$ whereever defined (!!) In higher dimensions, there are simply more functions that have the same scaling, leading to your error.
A: $
\def\p{{\partial}}
\def\grad#1#2{\frac{\p #1}{\p #2}}
\def\c#1{\color{red}{#1}}
$Separate
$f$ into two components: $\;$ its length $(\lambda)$ and direction $(\phi)$
$$\eqalign{
\lambda^2 &= \|f\|^2 &\quad\implies\quad \lambda\,d\lambda = \c{f^Tdf} \\
\phi &= \lambda^{-1} f \\
}$$
Calculate the differential of $\phi$
$$\eqalign{
d\phi &= \lambda^{-1} df - \lambda^{-2}f\,{d\lambda} \\
 &= \lambda^{-1} df - \lambda^{-3}f\,{(\lambda\,d\lambda)} \\
 &= \lambda^{-1}I\, df - \lambda^{-3}f\,\c{(f^Tdf)} \\
 &= \lambda^{-1}\left(I - \phi\phi^T\right)df \\
}$$
Now differentiate with respect to $x$
$$\eqalign{
\grad{\phi}{x} 
 &= \left(\frac{I - \phi\phi^T}{\lambda}\right)\grad{f}{x} \\
}$$
So your initial solution was absolutely correct.
And you can prove the famous result about unit vectors being perpendicular to their gradients
$$\eqalign{
\phi^T\left(\grad{\phi}{x}\right) 
 &= \left(\frac{\phi^T-({\tt1})\phi^T}{\lambda}\right)\grad{f}{x} \\
 &= \left(\frac{0}{\lambda}\right)\grad{f}{x} \\
 &= 0 \\
}$$
