Derivative of $f(x) = e^{a^Tx}$ Let $f(x) = e^{a^Tx}$ where $a,x \in \mathbb{R}^N$
Is $\frac {\partial f}{\partial x} = e^{a^Tx}a$ ?
 A: Consider the derivative with respect to $x_i$:
$$
\frac{\partial f}{\partial x_i} = \exp\left(\sum_{j \neq i}a_jx_j\right)a_i \exp\left(a_i x_i\right) = a_i\exp\left(\sum_{j}a_jx_j\right) = a_i e^{a^Tx}
$$
Since the derivative is most often denoted as the transpose of the gradient, we have
$$
Df = a^T e^{a^Tx} \\
\nabla f = ae^{a^Tx} .
$$
A: If $a \in R^N$, $x \in R^N$, we can write $x = (x_1, x_2, . . . , x_N)$, $a = (a_1, a_2, . . . , a_N)$ in the standard coordinate system.  Denoting the transpose of $a$ by $a^T$, we then have 
$a^Tx = \sum_{i = 1}^{i = N}a_ix_i$,
so that
$f(x) = \exp(\sum_{i=1}^{i = N}a_ix_i)$.
Then for any $x_j$,
$\frac{\partial f}{\partial x_j} = \exp(\sum_{i =1}^{i = N}a_ix_i)a_j$.
Thus
$\nabla f = \exp(\sum_{i = 1}^{i = N}a_ix_i)(a_1, a_2, . . . , a_N) = \exp(\sum_{i = 1}^{i = N}a_ix_i)a$.
This gives the most general derivative of $f(x)$ in the classic sense; I have occasionally seen the notation $\frac{\partial f}{\partial x}$ used to denote $\nabla f$. 
An alternative to the above view may be had by considering $Df(x)$ to be the linear map
such that
$f(x + h) = f(x) + Df(x)h + e(h)$,
for $h \in R^N$, where $e(h)$ is an error term with the property that $\frac{||e(h)||}{||h||} \to 0$ as $||h|| \to 0$.  Then since
$f(x + h) = \exp(\sum_{i = 1}^{i = N}a_i(x_i + h_i)) = \exp(\sum_{i = 1}^{i = N}a_ix_i)\exp(\sum_{i = 1}^{i = N}a_ih_i)$, 
and $\exp(\sum_{i = 1}^{i = N}a_ih_i)$ may be expanded in the usual power series
$\exp(\sum_{i = 1}^{i = N}a_ih_i) = 1 + \sum_{i = 1}^{i = N}a_ih_i + \frac{1}{2!}(\sum_{i = 1}^{i = N}a_ih_i)^2 + . . . $,
it is easily seen that 
$f(x + h) = \exp(\sum_{i=1}^{i = N}a_ix_i) + \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i + e(h)$
$= f(x) + \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i + e(h)$
showing that
$Df(x)h = \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i = \exp(\sum_{i=1}^{i = N}a_ix_i)a^Th = f(x)a^Th$,
whence
$Df(x) = \exp(\sum_{i=1}^{i = N}a_ix_i)a^T = f(x)a^T$.
I have also occasionally seen $\frac{\partial f}{\partial x}$ used in this context as well.
The difference between $\nabla f$ and $Df$ is, of course, that the former is technically a vector field while the latter is a one-form field (that is, a field of linear functionals on the tangent spaces $T_xR^N$).  But in Euclidean space, where forms and vectors are easily identified, the distinction twixt $\nabla f$ and $Df(x)$ is often overlooked.
