How to partially differentiate w.r.t. a vector? I have to prove that for two vectors $w$ and $b$ the following identity holds true
$$\frac{\partial(w^Tb)}{\partial w} = \frac{\partial(b^Tw)}{\partial w} = b$$
But I can't seem to understand how can I partially differentiate a function w.r.t. a vector? Does it mean that I differentiate each of its terms? As in if $w = [x_1, x_2, x_3]$ then does
$$\frac{\partial{x}}{\partial{w}} = \frac{\partial{}}{\partial{x_1}}\left(\frac{\partial{}}{\partial{x_2}}\left(\frac{\partial{x}}{\partial{x_3}}\right)\right)?$$
 A: There is no such thing as "partial differentiation with respect to a vector variable". Given a function $f:\>{\mathbb R}^n\to{\mathbb R}$,  a point $p$ in the domain of $f$, and a tangential vector $w\in T_p$  there is the Gâteaux derivative
$$D_w f(p):=\lim_{t\to0},{f(p+t w)-f(p)\over t}=df(p).w\ ,$$
also called the directional derivative of $f$ in direction $w$.
If $x$ is the declared coordinate variable in ${\mathbb R}^n$ then $df(p)$, being a functional on $T_p$, appears as
$$df(p).X=a_1X_1+\ldots a_nX_n=\nabla f(p)\cdot X$$ with
$$a_i={\partial f\over\partial x_i}(p)\qquad(1\leq i\leq n)\ .$$
The vector
$$\nabla f(p)=\left({\partial f\over\partial x_1},\ldots,{\partial f\over\partial x_n}\right)_p$$
is called the gradient of $f$ at $p$, and is, sometimes, denoted by ${\partial f \over\partial x}(p)$.
A: It means the gradient:
$$
\frac{\partial f}{\partial w} = \left( \frac{\partial f}{\partial w_1} , \dots, \frac{\partial f}{\partial w_n} \right)
.
$$
(If $\partial(w^T b)/\partial w$ is going to equal $b$, it had better be a vector, right?)
