let $f: \mathbb{R}^{n} \to \mathbb{R}^{m}$ be differentiable and $y \in \mathbb{R}^{m}$ be fixed

Define $g: \mathbb{R}^{n} \to \mathbb{R}$ as $g(x) = \|f(x) - y\|^2$

Now, I want to find $\nabla g$

Okay so, $g(x) = \langle f(x)- y, f(x) - y\rangle$ using properties of inner product we get

$g(x) = \langle f(x), f(x)\rangle - 2 \langle f(x), y\rangle + \langle y,y\rangle$

Now the derivative of $g$ at point $x\in \mathbb{R}^{n}$ is defined as :

$Dg_x = 2\langle Df_x, f\rangle - 2 \langle Df_x , y\rangle$

I am confused at this point and don't know how to get the gradient from here.

Any advice on how to proceed?

Thank you.


2 Answers 2


Take $p(x) = f(x) - y$ and $h(x) = \|x\|^2$. Then, $g = h \circ p$. $$Dg_x = D(h\circ p)_x = Dh_{p(x)} \circ Dp_x$$ We know that $Dh_{p(x)}(h) = 2\langle p(x),h \rangle$. $Dp_{x} = Df_x$ since $y$ is constant.

$$Dg_x(h) = Dh_{p(x)} \circ Dp_x(h) = 2\langle f(x) - y, f'(x)h\rangle = \langle \nabla g(x), h \rangle$$ for all $h$.

Can you find $\nabla g(x)$ now?

  • $\begingroup$ Yeah sorry that was a typo, we need to take that $f'(x)$ out of the inner product to get final expression. $\endgroup$
    – zeroflank
    Sep 5, 2021 at 17:04


If you are interested in the particular case where we are dealing with the Euclidian norm, we have: \begin{align*} g(x) & = \|f(x) - y\|^{2}\\\\ & = \|(f_{1}(x),f_{2}(x),\ldots,f_{m}(x)) - (y_{1},y_{2},\ldots,y_{m})\|^{2}\\\\ & = \|(f_{1}(x) - y_{1},f_{2}(x) - y_{2},\ldots,f_{m}(x) - y_{m})\|^{2}\\\\ & = (f_{1}(x) - y_{1})^{2} + (f_{2}(x) - y_{2})^{2} + \ldots + (f_{m}(x) -y_{m})^{2} \end{align*}

Can you take it from here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.