2
$\begingroup$

I have to prove that for two vectors $w$ and $b$ the following identity holds true

$$\frac{\partial(w^Tb)}{\partial w} = \frac{\partial(b^Tw)}{\partial w} = b$$

But I can't seem to understand how can I partially differentiate a function w.r.t. a vector? Does it mean that I differentiate each of its terms? As in if $w = [x_1, x_2, x_3]$ then does

$$\frac{\partial{x}}{\partial{w}} = \frac{\partial{}}{\partial{x_1}}\left(\frac{\partial{}}{\partial{x_2}}\left(\frac{\partial{x}}{\partial{x_3}}\right)\right)?$$

$\endgroup$
0
$\begingroup$

It means the gradient: $$ \frac{\partial f}{\partial w} = \left( \frac{\partial f}{\partial w_1} , \dots, \frac{\partial f}{\partial w_n} \right) . $$ (If $\partial(w^T b)/\partial w$ is going to equal $b$, it had better be a vector, right?)

$\endgroup$
0
$\begingroup$

There is no such thing as "partial differentiation with respect to a vector variable". Given a function $f:\>{\mathbb R}^n\to{\mathbb R}$, a point $p$ in the domain of $f$, and a tangential vector $w\in T_p$ there is the Gâteaux derivative $$D_w f(p):=\lim_{t\to0},{f(p+t w)-f(p)\over t}=df(p).w\ ,$$ also called the directional derivative of $f$ in direction $w$. If $x$ is the declared coordinate variable in ${\mathbb R}^n$ then $df(p)$, being a functional on $T_p$, appears as $$df(p).X=a_1X_1+\ldots a_nX_n=\nabla f(p)\cdot X$$ with $$a_i={\partial f\over\partial x_i}(p)\qquad(1\leq i\leq n)\ .$$ The vector $$\nabla f(p)=\left({\partial f\over\partial x_1},\ldots,{\partial f\over\partial x_n}\right)_p$$ is called the gradient of $f$ at $p$, and is, sometimes, denoted by ${\partial f \over\partial x}(p)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.