2
$\begingroup$

I understand the theory of symbolic and numerical differentiation pretty well, however am struggling on understanding automatic differentiation.

I have read the following example in documentation and struggle with how certain steps are performed and require clarity.

By way of example, say we want to find the solution (using automatic differentiation) of $f'(3)$ where $f(x)$ is equal to:

$$f(x) = \frac{(x+1)(x-2)}{x+3}$$

My understanding is to use automatic differentiation method, prior to solving for $f'(x)$, we must understand two things:

1) We must work with the concept of "value pairs" when evaluating expressions i.e. $$\vec{u} = (u, u')$$ where u denotes the value of the function at a point $x_{0}$

2) For the purposes of this simple example, understand the two symbolic differentiation rules: $$\vec{x} = (x, 1)$$ $$\vec{c} = (c, 0)$$

Now using the rules above we evaluate as follow's: Example

In that example, how did they get the value of 5 in the expression: $\frac{(4, 5)}{(6, 1)}$ ? and how did they get a result of 13/18?

$\endgroup$
1

2 Answers 2

2
$\begingroup$

You need to employ rules for addition, multiplication, etc. as obtained from the basic rules for differentiation, that is $$\begin{align} (a,a')+(b,b')&=(a+b,a'+b')\\ (a,a')-(b,b')&=(a-b,a'-b')\\ (a,a')\cdot (b,b')&=(ab,a'b+ab')\\ \frac{(a,a')}{(b,b')}&=\left(\frac ab,\frac{a'b-ab'}{b^2}\right)\\ \end{align} $$ Thus $$(4,1)\cdot (1,1)=(4\cdot 1,1\cdot 1+4\cdot 1)=(4,5). $$

$\endgroup$
3
  • $\begingroup$ Thanks, they broke this up this step in the documentation $\endgroup$
    – Dean P
    May 30, 2017 at 16:14
  • $\begingroup$ I've done a bit of reading and all sources say that AD is completely different to Symbolic differentiation. I see it as a subset. Ultimately, instead of manipulating the entire symbolic expression as a whole, the symbols in AD are manipulated individually so that complex symbolic rules are bypassed. Am I a bit off in my understanding? $\endgroup$
    – Dean P
    May 31, 2017 at 11:49
  • $\begingroup$ I am seeing in the literature that you can get away will only knowing the chain rule. How could that work in the above example and what are the implications? (longer computational time, etc...) $\endgroup$
    – Dean P
    May 31, 2017 at 13:08
1
$\begingroup$

The differentiation rule for a quotient is

$$\left(\frac pq\right)'=\frac{p'q-pq'}{p^2},$$

which can be written

$$\frac{(p,q)}{(r,s)}\to\left(\frac pr,\frac{qr-ps}{r^2}\right).$$

$\endgroup$
1
  • $\begingroup$ Thanks, they broke this up this step in the documentation $\endgroup$
    – Dean P
    May 30, 2017 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.