# Calculate the gradient of $g(x) = \|f(x) - y\|^2$

let $$f: \mathbb{R}^{n} \to \mathbb{R}^{m}$$ be differentiable and $$y \in \mathbb{R}^{m}$$ be fixed

Define $$g: \mathbb{R}^{n} \to \mathbb{R}$$ as $$g(x) = \|f(x) - y\|^2$$

Now, I want to find $$\nabla g$$

Okay so, $$g(x) = \langle f(x)- y, f(x) - y\rangle$$ using properties of inner product we get

$$g(x) = \langle f(x), f(x)\rangle - 2 \langle f(x), y\rangle + \langle y,y\rangle$$

Now the derivative of $$g$$ at point $$x\in \mathbb{R}^{n}$$ is defined as :

$$Dg_x = 2\langle Df_x, f\rangle - 2 \langle Df_x , y\rangle$$

I am confused at this point and don't know how to get the gradient from here.

Any advice on how to proceed?

Thank you.

Take $$p(x) = f(x) - y$$ and $$h(x) = \|x\|^2$$. Then, $$g = h \circ p$$. $$Dg_x = D(h\circ p)_x = Dh_{p(x)} \circ Dp_x$$ We know that $$Dh_{p(x)}(h) = 2\langle p(x),h \rangle$$. $$Dp_{x} = Df_x$$ since $$y$$ is constant.

$$Dg_x(h) = Dh_{p(x)} \circ Dp_x(h) = 2\langle f(x) - y, f'(x)h\rangle = \langle \nabla g(x), h \rangle$$ for all $$h$$.

Can you find $$\nabla g(x)$$ now?

• Yeah sorry that was a typo, we need to take that $f'(x)$ out of the inner product to get final expression. Sep 5, 2021 at 17:04

HINT

If you are interested in the particular case where we are dealing with the Euclidian norm, we have: \begin{align*} g(x) & = \|f(x) - y\|^{2}\\\\ & = \|(f_{1}(x),f_{2}(x),\ldots,f_{m}(x)) - (y_{1},y_{2},\ldots,y_{m})\|^{2}\\\\ & = \|(f_{1}(x) - y_{1},f_{2}(x) - y_{2},\ldots,f_{m}(x) - y_{m})\|^{2}\\\\ & = (f_{1}(x) - y_{1})^{2} + (f_{2}(x) - y_{2})^{2} + \ldots + (f_{m}(x) -y_{m})^{2} \end{align*}

Can you take it from here?