2
$\begingroup$

Let $f(x) = e^{a^Tx}$ where $a,x \in \mathbb{R}^N$

Is $\frac {\partial f}{\partial x} = e^{a^Tx}a$ ?

$\endgroup$
9
  • $\begingroup$ what is $T$??${{}}$ $\endgroup$ Jul 17, 2013 at 16:13
  • $\begingroup$ I'm assuming $a^T$ is a real number and that $f$ is a real function of real variable. Your notation $\dfrac{\partial f(x)}{\partial x}$ is incorrect. It should be either $\dfrac{\partial f}{dx}(x)$ or $\dfrac{\partial f}{dx}$, depending on what you'd write on the other side of the equality. $\endgroup$
    – Git Gud
    Jul 17, 2013 at 16:14
  • $\begingroup$ @MonkeyD.Luffy Sorry $T$ is transpose notation. $a$ and $x$ are vectors. $\endgroup$
    – Rein
    Jul 17, 2013 at 16:19
  • $\begingroup$ @GitGud May I know what are the differences? $\endgroup$
    – Rein
    Jul 17, 2013 at 16:20
  • $\begingroup$ @Rein You should add that info to the question. $\endgroup$
    – Git Gud
    Jul 17, 2013 at 16:24

2 Answers 2

3
$\begingroup$

Consider the derivative with respect to $x_i$: $$ \frac{\partial f}{\partial x_i} = \exp\left(\sum_{j \neq i}a_jx_j\right)a_i \exp\left(a_i x_i\right) = a_i\exp\left(\sum_{j}a_jx_j\right) = a_i e^{a^Tx} $$

Since the derivative is most often denoted as the transpose of the gradient, we have $$ Df = a^T e^{a^Tx} \\ \nabla f = ae^{a^Tx} . $$

$\endgroup$
0
$\begingroup$

If $a \in R^N$, $x \in R^N$, we can write $x = (x_1, x_2, . . . , x_N)$, $a = (a_1, a_2, . . . , a_N)$ in the standard coordinate system. Denoting the transpose of $a$ by $a^T$, we then have

$a^Tx = \sum_{i = 1}^{i = N}a_ix_i$,

so that

$f(x) = \exp(\sum_{i=1}^{i = N}a_ix_i)$.

Then for any $x_j$,

$\frac{\partial f}{\partial x_j} = \exp(\sum_{i =1}^{i = N}a_ix_i)a_j$.

Thus

$\nabla f = \exp(\sum_{i = 1}^{i = N}a_ix_i)(a_1, a_2, . . . , a_N) = \exp(\sum_{i = 1}^{i = N}a_ix_i)a$.

This gives the most general derivative of $f(x)$ in the classic sense; I have occasionally seen the notation $\frac{\partial f}{\partial x}$ used to denote $\nabla f$.

An alternative to the above view may be had by considering $Df(x)$ to be the linear map such that

$f(x + h) = f(x) + Df(x)h + e(h)$,

for $h \in R^N$, where $e(h)$ is an error term with the property that $\frac{||e(h)||}{||h||} \to 0$ as $||h|| \to 0$. Then since

$f(x + h) = \exp(\sum_{i = 1}^{i = N}a_i(x_i + h_i)) = \exp(\sum_{i = 1}^{i = N}a_ix_i)\exp(\sum_{i = 1}^{i = N}a_ih_i)$,

and $\exp(\sum_{i = 1}^{i = N}a_ih_i)$ may be expanded in the usual power series

$\exp(\sum_{i = 1}^{i = N}a_ih_i) = 1 + \sum_{i = 1}^{i = N}a_ih_i + \frac{1}{2!}(\sum_{i = 1}^{i = N}a_ih_i)^2 + . . . $,

it is easily seen that

$f(x + h) = \exp(\sum_{i=1}^{i = N}a_ix_i) + \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i + e(h)$ $= f(x) + \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i + e(h)$

showing that

$Df(x)h = \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i = \exp(\sum_{i=1}^{i = N}a_ix_i)a^Th = f(x)a^Th$,

whence

$Df(x) = \exp(\sum_{i=1}^{i = N}a_ix_i)a^T = f(x)a^T$.

I have also occasionally seen $\frac{\partial f}{\partial x}$ used in this context as well.

The difference between $\nabla f$ and $Df$ is, of course, that the former is technically a vector field while the latter is a one-form field (that is, a field of linear functionals on the tangent spaces $T_xR^N$). But in Euclidean space, where forms and vectors are easily identified, the distinction twixt $\nabla f$ and $Df(x)$ is often overlooked.

$\endgroup$
2
  • $\begingroup$ Downvoted, and at this late date, and with no explanation! The downvote gremlins are out to get me! $\endgroup$ Mar 20, 2014 at 4:02
  • $\begingroup$ Would that these mystery downvoters possessed the huevos to name themselves and put forth (hopefully) mathematical reasons for their choices. May they "die many times before their deaths!" I'll die but once, thank you. $\endgroup$ Mar 20, 2014 at 4:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.