Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
Suppose $~W~$ is an $~n \times n~$ matrix and $~x$,$~y~$ are $~n \times 1~$ vectors. Then define function $~f~$ as following:
\begin{equation} f=\left\|W\left(x-y\right)\right\|^{2} \end{equation} The question is what are $\frac{\partial f}{\partial W}$ and $\frac{\partial f}{\partial x}$?
Use chain rule and verify the following facts:
For Euclidean vector norm $\| \cdot \| : \mathbb{R^{n}} \to \mathbb{R}$:
$$ \frac{d \| \cdot \|}{d \mathbf{x}} = \frac{\mathbf{x}}{\|x\|} ; $$
For $f:\mathbb{R}\to \mathbb{R}$, $f(t) := t^2$ the derivative is: $$ \frac{d f(t) }{dt} = \frac{d t^2 }{dt} = 2t; $$
For $g:\mathbb{R^n}\to \mathbb{R^n}$, $~g(\mathbf{x}) := \mathbf{W}\mathbf{x}$:
$$ \frac{d g(\mathbf{x}) }{d\mathbf{x}} = \frac{d \mathbf{W}\mathbf{x} }{d\mathbf{x}} = \mathbf{W}; $$
For $h:\mathbb{R^{n\times n}}\to \mathbb{R^n}$, $~h(\mathbf{W}) := \mathbf{W}\mathbf{x}$:
$$ \frac{d h(\mathbf{W}) }{d\mathbf{W}} = \frac{d \mathbf{W}\mathbf{x} }{d\mathbf{W}} = \mathbf{x}^\top. $$
