I have the formula, and I was reading a basic implementation of automatic differentiation:

$$f\left(x\right) = x - \exp\left(-2\sin^{2}\left(4x\right)\right).$$

The evaluation trace for

$$f(\pi/16)$$ and $$f'(\pi/16)$$

where $f^{\prime}\left(x\right) = \dfrac{\partial f}{\partial x}$.

looks like this:

enter image description here

My question is, I do not understand how the derivatives are calculated (meaning the 3rd and fourth columns). Maybe I am missing something simply here, but the derivative of sin(4x) is not cos(x)(x), for example.


The third row, for example should be read as:

  • Set $x_3 = \sin(x_2)$
  • Then $\mathrm{d}x_3 = \cos(x_2) ~\mathrm{d}x_2$.
  • Now combine it with the previous line where $x_2$ has been computed to equal $\frac{\pi}{4}$ and $\mathrm{d}x_2$ has computed to be equal to $4$, then $x_3 = \sin(\pi/4)$ and $\mathrm{d}x_3 = \cos(\pi/4) \cdot 4$.

(note that in automatic differentiation the object $\mathrm{d}x_2$ is not an infinitesimal differential, but an element of $T_{x_2} \mathbb{R}$, the tangent space of $\mathbb{R}$ at $x_2$.)

Another way of thinking about it is that the table is just a very verbose way of writing out the chain rule of differentiation; except unlike what you are familiar with doing in calculus class, it starts from the inside-most term and move steadily outward.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.