Given the function:

$$y({\bf w}) = \| \,{\bf w\, w^T x - x } \,\|^2_2$$

I'm trying to understand how to get the derivative $\frac{\partial y}{\partial\bf w}$, where $\bf w$ and $\bf x$ are vectors.

Any help would be much appreciated.


1 Answer 1


Hints: $y$ is a composition of two functions, $w \mapsto w w^T x-x$ and then $w \mapsto \|w\|_2^2$. Let me remark that $$ w w^T x - x = \langle w,x\rangle w - x, $$ where $\langle -,- \rangle$ denotes the $L^2$-inner product. To apply the chain rule, you should first compute $$ \frac{\partial}{\partial w} \left( \langle w,x\rangle w - x \right) = \langle -,x \rangle w + \langle w,x \rangle \operatorname{I}, $$ where I is the identity operator. Finally, you want the derivative of the $L^2$ squared norm. This is easy, since it comes from a scalar product: $$ \frac{\partial}{\partial w} \|w\|_2^2 = \frac{\partial}{\partial w} \langle w,w \rangle = 2 \langle -,w \rangle. $$

  • $\begingroup$ This looks promising but I'm a little confused by the terminology. By $\langle -,w\rangle$ do you mean the inner product of $w$ with something unknown? I guess I'm still confused as to how the two partial results tie together in your terminology. $\endgroup$
    – Robotbugs
    Aug 14, 2015 at 1:05
  • $\begingroup$ If I wanted $\frac{\partial}{\partial w_i}$ where $w_i$ is an element of $w$ does that make it clearer? I'm confused about the result being an operator. $\endgroup$
    – Robotbugs
    Aug 14, 2015 at 1:17
  • $\begingroup$ The derivative is an operator that acts on a generic element of the space. By $\langle -,w \rangle$ I mean the operator $h \mapsto \langle h,w \rangle$. $\endgroup$
    – Siminore
    Aug 14, 2015 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.