Could someone please provide a proof for why the gradient of the squared $2$-norm of $x$ is equal to $2x$?

$$\nabla\|x\|_2^2 = 2x$$


3 Answers 3


Use the definition. If $$f(x)=\|x\|^2_2= \left(\left(\sum_{k=1}^n x_k^2 \right)^{1/2}\right)^{2}=\sum_{k=1}^n x_k^2 ,$$ then $$\frac{\partial}{\partial x_j}f(x) =\frac{\partial}{\partial x_j}\sum_{k=1}^n x_k^2=\sum_{k=1}^n \underbrace{\frac{\partial}{\partial x_j}x_k^2}_{\substack{=0, \ \text{ if } j \neq k,\\=2x_j, \ \text{ else }}}= 2x_j.$$ It follows that $$\nabla f(x) = 2x.$$

  • $\begingroup$ Thanks. One follow up question, does this still hold if x is complex? $\endgroup$
    – user167133
    Jul 30, 2014 at 20:56
  • $\begingroup$ Isn't the L2 norm defined as: $$f(x)=\|x\|^2_2= \left(\left(\sum_{k=1}^n |x_k|^2 \right)^{1/2}\right)^{2}=\sum_{k=1}^n |x_k|^2$$, there is an absolute value sign surrounding $x_k$, so when you take the derivative what should pop out is $2|x_j|$ not just $2x_j$, correct me if I am wrong $\endgroup$
    – Fraïssé
    Oct 23, 2015 at 17:46
  • 4
    $\begingroup$ @Lookbehindyou well |t|^2 = t^2 for any $t\in\Bbb R$. Moreover, note that $\frac{d}{t}|t|^2 = 2|t|\operatorname{sign}(t)=2t$, where $\operatorname{sign}$ is the sign function $\endgroup$
    – Surb
    Feb 29, 2016 at 14:49
  • 1
    $\begingroup$ Could you please explain what happened to the sigma in the last step? $\endgroup$
    – Gigili
    Aug 16, 2016 at 12:38
  • 1
    $\begingroup$ @Gigili: If $k\neq j$, then $\frac{\partial }{\partial x_j}x_k^2=0$. Then, $$\sum_{k=1}^n\frac{\partial }{\partial x_j}x_k^2=0+...+0+2x_j+0+...+0=2x_j.$$ $\endgroup$
    – Surb
    Aug 16, 2016 at 12:44

Another approach that extends to more general settings is to use the connection between the norm and the inner product, $$\|x\|^2 = (x,x).$$

We have the finite difference, \begin{align} \|x+sh\|^2 - \|x\|^2 &= (x+sh,x+sh) - (x,x) \\ &= (x,x) + 2s(x,h) + s^2(h,h) - (x,x) \\ &= 2s(x,h) + s^2(h,h). \end{align}

The gradient acting in the direction $h$ is the limit of this finite difference as the stepsize goes to zero, \begin{align} (\nabla\|x\|^2, h) &:= \lim_{s \rightarrow 0} \frac{1}{s}\left[\|x+sh\|^2 - \|x\|^2\right] \\ &= \lim_{s \rightarrow 0} \frac{1}{s}\left[2s(x,h) + s^2(h,h)\right] \\ &= (2x,h). \end{align} Since this holds for any direction $h$, the gradient must be $\nabla \|x\|^2 = 2x$.


I'm not sure if this is rigorous enough to count as a proof, but an elegant way to obtain derivatives of vector expressions is to use matrix differential calculus.

Let $y = \lVert x \rVert_2^2 = x^{T} x$ with $x \in \mathbb{R}^{n}$. Using the product rule, the differential of $y$ is $$ dy = dx^{T} x + x^{T} dx = 2 x^{T} dx $$

We can then set $$ dy = \frac{dy}{dx} dx = (\nabla_{x} y)^{T} dx = 2x^{T} dx $$ where $dy/dx \in \mathbb{R}^{1 \times n}$ is called the derivative (a linear operator) and $\nabla_{x} y \in \mathbb{R}^{n}$ is called the gradient (a vector).

Now we can see $\nabla_{x} y = 2 x$.

If $x$ is complex, the complex derivative does not exist because $z \mapsto |z|^{2}$ is not a holomorphic function.

We can, however, instead consider the real derivatives with respect to the two components of $x$. Let $x = u + i v$. With this definition, $y$ is a real function of $u, v \in \mathbb{R}^{n}$ defined by $$ y = x^* x = (u + i v)^* (u + i v) = u^T u - v^T v $$ Taking the differential $$ dy = 2 u^T du - 2 v^T dv = \frac{\partial y}{\partial u} du + \frac{\partial y}{\partial v} dv $$ and therefore $$ \nabla_{u} y = 2 u \enspace , \qquad \nabla_{v} y = -2 v $$

For an introduction to matrix differential calculus, see the lecture of Geoff Gordon on YouTube or the paper on matrix derivatives of Mike Giles.

  • $\begingroup$ sorry me, why $x^T dx + dx^T x = 2x^T dx$ ? I suppose, that dx^T = dx, because it is number, but what we can do with x + x^T .. $\endgroup$
    – GThompson
    Nov 7, 2021 at 23:13
  • $\begingroup$ Actually $dx$ has the same dimension as $x$, both are in $R^n$ $\endgroup$ Nov 9, 2021 at 7:56
  • $\begingroup$ In case it wasn't clear - for $a, b \in R^n$, we have $a^T b = b^T a = \sum_i a_i b_i$ $\endgroup$ May 3 at 8:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.