1
$\begingroup$

I have encounted two methods for the calculation of second derivatives via foward mode AD.

Fike, Jeffrey, and Juan Alonso. "The development of hyper-dual numbers for exact second-derivative calculations." 49th AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition. 2011

and Chapter 13 of the textbook:
Griewank, Andreas, and Andrea Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation. Vol. 105. Siam, 2008.
(I think this was also in the 1987 edition but I don't have a copy handy)

Fike and Alonso define a new extension of Dual numbers, with 3 terms. And define numerical operations on those, and show that by propagating them forward through a function, its first and second deriviative can be found.

Griewank and Walther give standard numerical operations on Taylor polynomials, and show that by propagating a Taylor polynomial of degree $d$ forward, through a function, all its deriviative's up to order $d$ can be found.

I don't understand either method in detail.
But is it known that the hyper-dual method is a special case of the Taylor polynomial method?
They seem quiet similar.

However, Fike and Alonso do cite Griewank and Walther 2008, for some other points. So I would have expected if their method was very similar for them to point it out.

$\endgroup$
2
  • 1
    $\begingroup$ Dual numbers are the same as arithmetic on power series, truncated to a linear term. The paper you link to just iterates the construction of dual numbers (i.e. essentially adds a new independent variable along which change can be measured), which leads to using four terms (two of them equal!) to represent 3 term power series - which make those hyper-dual numbers look like more redundant Taylor series. (...honestly, the hyper-dual number paper reads to me like the authors did not understand how Taylor series work or why the dual number construction works...) $\endgroup$ Commented Jan 6, 2020 at 22:22
  • $\begingroup$ Its important to realize, and the paper doesn;'t make it clear, that for $a+bε_1+cε_2+d_3$ that always $b=c$ since both are equal to something's first derivative $\endgroup$ Commented Jan 14, 2020 at 22:54

1 Answer 1

0
$\begingroup$

I am not going to do a rigorous proof. But what follows is a demonstration that there is a mapping between any hyperduel and any degree 2 polynomial such that multiplication results the same. This can be extended to other operations.

Basic definitions:

Hyperduels:

$$a=a_{0}+a_{1}\epsilon_{1}+a_{2}\epsilon_{2}+a_{3}\epsilon_{1}\epsilon_{2}$$ $$b=b_{0}+b_{1}\epsilon_{1}+b_{2}\epsilon_{2}+b_{3}\epsilon_{1}\epsilon_{2}$$ $$ab=a_{0}b_{0}+\left(a_{1}b_{0}+a_{0}b_{1}\right)\epsilon_{1}+\left(a_{2}b_{0}+a_{0}b_{2}\right)\epsilon_{2}+\left(a_{0}b_{3}+a_{1}b_{2}+a_{2}b_{1}+a_{3}b_{0}\right)\epsilon_{1}\epsilon_{2}$$

Polynomials of degree 2:

$$w=w_{0}+w_{1}x^{1}+w_{2}x^{2}$$ $$u=u_{0}+u_{1}x^{1}+u_{2}x^{2}$$ $$uw=w_{0}u_{0}+\left(w_{0}u_{1}+w_{1}u_{0}\right)x^{1}+(w_{0}u_{2}+w_{1}u_{1}+w_{2}u_{0})x^{2}$$

The first thing to note is: The only use of hyperduels uses in differentiation: $a_{1}=a_{2}$, and $b_{1}=b_{2}$ since they are both equal to first derivative of something (the initial perturbation).

(If you do not hold this then hyperduels and polynomials of degree 2 are not the same thing. Since one has a extra free variable)

Thus actually: $$a=a_{0}+a_{1}\epsilon_{1}+a_{1}\epsilon_{2}+a_{3}\epsilon_{1}\epsilon_{2}$$

Thus:

$$ab=a_{0}b_{0}+\left(a_{1}b_{0}+a_{0}b_{1}\right)\epsilon_{1}+\left(a_{1}b_{0}+a_{0}b_{1}\right)\epsilon_{2}+\left(a_{0}b_{3}+2a_{1}b_{1}+a_{3}b_{0}\right)\epsilon_{1}\epsilon_{2}$$

Now:

We declare a mapping between Polynomials of Degree 2 and HyerDuels

$$h\mapsto h'$$

$$h_{0}+h_{1}x+h_{2}x^{2}\mapsto h_{0}+h_{1}\epsilon_{1}+h_{1}\epsilon_{2}+2(h_{2}-h_{1})\epsilon_{1}\epsilon_{2}$$

Under this:

$$w^{\prime}=w_{0}+w_{1}\epsilon_{1}+w_{1}\epsilon_{2}+2(w_{2}-w_{1})\epsilon_{1}\epsilon_{2}$$

$$u^{\prime}=u_{0}+u_{1}\epsilon_{1}+u_{1}\epsilon_{2}+2(u_{2}-u_{1})\epsilon_{1}\epsilon_{2}$$

$$w^{\prime}u^{\prime}=w_{0}u_{0}+\left(w_{0}u_{1}+w_{1}u_{0}\right)\epsilon_{1}+\left(w_{0}u_{1}+w_{1}u_{0}\right)\epsilon_{2}+\left(2\left(u_{2}w_{0}-u_{1}w_{0}\right)+2u_{1}w_{1}+2\left(w_{2}u_{0}-w_{1}u_{0}\right)\right)\epsilon_{1}\epsilon_{2}$$

$$w^{\prime}u^{\prime}=w_{0}u_{0}+\left(w_{0}u_{1}+w_{1}u_{0}\right)\epsilon_{1}+\left(w_{0}u_{1}+w_{1}u_{0}\right)\epsilon_{2}+2\left(\left(u_{2}w_{0}+u_{1}w_{1}+w_{2}u_{0}\right)-(w_{1}u_{0}+u_{1}w_{0}\right)\epsilon_{1}\epsilon_{2}$$

taking the mapping back the other way we see:

$$\left(w^{\prime}u^{\prime}\right)^{-\prime}=w_{0}u_{0}+\left(w_{0}u_{1}+w_{1}u_{0}\right)x+\left(u_{2}w_{0}+u_{1}w_{1}+w_{2}u_{0}\right)x^{2}$$

Which is the same as: $$uw=w_{0}u_{0}+\left(w_{0}u_{1}+w_{1}u_{0}\right)x^{1}+(w_{0}u_{2}+w_{1}u_{1}+w_{2}u_{0})x^{2}$$ which we saw before.

One can go through and show they have matching addition operations similarly.

And then show their multiplicative and additive identities are equal.

Until you have demonstrated they are the same thing for what ever definition of same you require.

$\endgroup$

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.