Given the function:
$$y({\bf w}) = \| \,{\bf w\, w^T x - x } \,\|^2_2$$
I'm trying to understand how to get the derivative $\frac{\partial y}{\partial\bf w}$, where $\bf w$ and $\bf x$ are vectors.
Any help would be much appreciated.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Hints: $y$ is a composition of two functions, $w \mapsto w w^T x-x$ and then $w \mapsto \|w\|_2^2$. Let me remark that $$ w w^T x - x = \langle w,x\rangle w - x, $$ where $\langle -,- \rangle$ denotes the $L^2$-inner product. To apply the chain rule, you should first compute $$ \frac{\partial}{\partial w} \left( \langle w,x\rangle w - x \right) = \langle -,x \rangle w + \langle w,x \rangle \operatorname{I}, $$ where I is the identity operator. Finally, you want the derivative of the $L^2$ squared norm. This is easy, since it comes from a scalar product: $$ \frac{\partial}{\partial w} \|w\|_2^2 = \frac{\partial}{\partial w} \langle w,w \rangle = 2 \langle -,w \rangle. $$
-
$\begingroup$ This looks promising but I'm a little confused by the terminology. By $\langle -,w\rangle$ do you mean the inner product of $w$ with something unknown? I guess I'm still confused as to how the two partial results tie together in your terminology. $\endgroup$ Aug 14, 2015 at 1:05
-
$\begingroup$ If I wanted $\frac{\partial}{\partial w_i}$ where $w_i$ is an element of $w$ does that make it clearer? I'm confused about the result being an operator. $\endgroup$ Aug 14, 2015 at 1:17
-
$\begingroup$ The derivative is an operator that acts on a generic element of the space. By $\langle -,w \rangle$ I mean the operator $h \mapsto \langle h,w \rangle$. $\endgroup$– SiminoreAug 14, 2015 at 9:51