# Gradient of a norm with a linear operator

In mathematical image processing many algorithms are stated as an optimization problem, where we have an observation $f$ and want recover an image $u$ that minimizes a objective function. Further, to gain smooth results a regularization term is applied to the image gradient $\nabla u$, which can be implemented with finite differences in $x, y$ direction.

If I have a regularizer of the form $||\nabla u||_2$, with $||x||_2 = \sqrt{\sum_i x_i^2}$ how can I compute the gradient with respect to $u$? I don't know how to handle the operator $\nabla$.

• Do the same thing you would in multivariable calculus, the gradient "in the $v$ direction" of $\nabla u$ is equal to $\lim_{s \to 0} s^{-1}\left[ \nabla (u + sv) = \nabla u\right] = \nabla v$. Now similarly you can compute $\lim_{s \to 0} s^{-1} \left[ \|\nabla (u+sv)\|_2 - \|\nabla u\|_2 \right]$ for the gradient of the regularizer. – Willie Wong Dec 6 '13 at 9:41

$|| \nabla u ||_2 = || \sqrt{ (u^x)^2 + (u^y)^2 } ||_2 = || \sqrt{ (u_{i+1,j} - u_{i,j})^2 + (u_{i,j+1} - u_{i,j})^2 } ||_2$
where it is now easy to take the derivative with respect to $u_{i,j}$.
Note also that $|| \nabla u ||_2$ implicitly means $|| \nabla u ||_{2,2}$.