41
$\begingroup$

I've been interested in non-standard analysis recently. I was reading up on it and noticed the following interesting comment on the Wikipedia page about hyperreal numbers, right after giving an example of a nonstandard differentiation:

The use of the standard part in the definition of the derivative is a rigorous alternative to the traditional practice of neglecting the square of an infinitesimal quantity... the typical method from Newton through the 19th century would have been simply to discard the $dx^2$ term.

I've never heard anything like this before, and really find it fascinating that Newton's method was to define the relation $dx^2 = 0$. If we actually formalize the above structure by taking $\mathbb{R}$ and adjoining an element $dx^2 = 0$ to it, we get the "dual numbers," isomorphic to the quotient ring $\mathbb{R}[x]/x^2$. I'd seen some things about how this algebra plays into automated differentiation algorithms for some computer software systems, but I've never heard anything about Newton directly working in this algebra. So I have a few questions:

  1. Does anyone have more historical information on the way that Newton performed differentiation, and its relation to the dual numbers?
  2. Does anyone know how effectively real analysis can be formalized with the dual numbers? Does the resulting system play nice enough to develop all of the important modern results?
  3. If we start with $\mathbb{C}[x]/x^2$ instead, can we likewise develop complex analysis?

Since this idea is so simple, I'm very curious how powerful it is. I'm also curious if it has any major drawbacks too, since I'm not sure why anyone would mess with the foundational baggage involved in defining the hyperreals if this simple 2-dimensional real algebra could really do the trick.

$\endgroup$
2
  • 1
    $\begingroup$ The "major drawback" as you put it is that an ordinary function on the reals does not extend to the dual numbers. $\endgroup$
    – Mikhail Katz
    Dec 19, 2014 at 9:02
  • $\begingroup$ I got so many comments about how 𝑑𝑥2=0 causes problems that I made a new question about automatic differentiation in ℝ((𝑑𝑥)), where 𝑑𝑥 is just a new formal quantity that isn't nilpotent. This seems to do better, since you get an expansion of derivatives of all orders, and you also get a field. See here: math.stackexchange.com/questions/3202846/… $\endgroup$
    – Mike Battaglia
    Nov 20, 2020 at 19:13

8 Answers 8

Reset to default
21
$\begingroup$

The biggest draw back (and it's a big one) is that the ring of dual numbers is not a field. It has plenty of zero divisors. So, Newton, or any of the mathematicians of the early days of calculus, certainly did not work directly in the ring of dual numbers. They of course did not consider the ring to exist (as rings did not exist at all yet), but from their writing it is clear they envisaged a field of real numbers with, somehow, some notions of infinitesimals. Their work is of course very vague, but correct. Much more on that can be found in math history books. Many interesting discussions can be found in the recent book "Adventures in Formalism", also related to the early days of calculus and how things developed.

Some (rather unsatisfactory) portions of analysis can be developed in the ring of dual numbers, but it does not go too far. The idea, as you say, is very simple, perhaps too simple. One immediately gets into trouble when trying to define the derivative as the quotient of the infinitesimal $f(x+h)-f(x)$ divided by $h$, where $h$ is infinitesimal. The difficulty is that the non-zero infinitesimals in the ring of dual numbers are not invertible. So, it's the end of the party. (As you say though, some aspects of the party remain with automatic differentiation). In some sense, the dual numbers form a first order approximation to actual infinitesimals: The square of an infinitesimal is of an order of magnitude smaller than the infinitesimal you started with, but in the ring of dual numbers, the square of an 'infinitesimal' is precisely $0$. So, in a nonstandard model of the reals you have whole layers of infinitesimals. In the dual numbers there is only one layer, nothing in it is invertible, and they all square to $0$.

The book Models for smooth infinitesimal analysis explores many different models for analysis with infinitesimals. None of them is particularly simple.

$\endgroup$
16
  • 10
    $\begingroup$ On the other hand, for a polynomial $f$ we have that $f(x+\varepsilon)-f(x)$ equals $f'(x) \cdot \varepsilon$, so that $f'(x)=\frac{f(x+\varepsilon)-f(x)}{\varepsilon}$ in $k[\varepsilon]/\varepsilon^2 [x]$. This also illustrates that the ring of dual numbers is useful in algebraic geometry. $\endgroup$
    – Martin Brandenburg
    Mar 26, 2013 at 9:13
  • 3
    $\begingroup$ Thanks for the detailed response and the reference, which I'll definitely check out. A question: is there any utility in adjoining an element $\omega = \frac{1}{\epsilon}$ to the ring, which then has the property that $\omega^2 = \infty$ (in the extended real line sense)? This still wouldn't give you a field, but it would at least make the various dual numbers invertible, and having derivatives take values in $\mathbb{R} \cup {\infty}$ doesn't seem that far out there. $\endgroup$
    – Mike Battaglia
    Mar 26, 2013 at 9:19
  • 2
    $\begingroup$ Quick afterthought since I can't edit: That should say $\mathbb{R} \cup \{\infty, -\infty\}$ above. So you'd get numbers of the form $a + b\epsilon + c\omega$, where $\epsilon^2 = 0$ and $\omega^2 = \infty$ and $-\omega^2 = -\infty$. Not a field, but still seems possibly useful... $\endgroup$
    – Mike Battaglia
    Mar 26, 2013 at 9:25
  • 1
    $\begingroup$ @MikeBattaglia, you will still have the problem of there not being any layers of infinitesimals. The dual number is a rather crude model for adjoining infinitesimals. As Martin explains in his answer the comment, the dual numbers do find applications in algebraic geometry. But for analysis, it does not look promising. $\endgroup$
    – Ittay Weiss
    Mar 26, 2013 at 9:32
  • 3
    $\begingroup$ The main problem with the dual numbers is not that they are not a field, but that an ordinary function does not extend to the dual numbers, and therefore this approach is not helpful in calculus and analysis. $\endgroup$
    – Mikhail Katz
    Dec 19, 2014 at 9:01
9
$\begingroup$

No for 1. and 3., this ring is not really useful in analysis. But it is quite important for analytical considerations in algebraic geometry, the main reason being that the scheme $\mathrm{Spec}(k[\varepsilon]/\varepsilon^2)$ classifies tangent vectors. This makes it possible to define the tangent space of arbitrary functors $F : \mathsf{CRing} \to \mathsf{Set}$ at some $x \in F(k)$, namely as the fiber of $F(k[\varepsilon]/\varepsilon^2) \to F(k)$ at $x$. There is no manifold which represents tangent vectors for manifolds, so this is the main difference.

$\endgroup$
4
  • 1
    $\begingroup$ This seems like a great response but unfortunately I'm not able to understand it yet! I've just started Hartshorne now but I'm still working on sheaves and haven't gotten to schemes yet. I'm going to have to come back to this in a few weeks and see if it all makes sense then.. $\endgroup$
    – Mike Battaglia
    Mar 26, 2013 at 9:23
  • $\begingroup$ Better look up the notion of a derivation and try to prove that homomorphisms of rings $A \to k[\varepsilon]/\varepsilon^2$ correspond 1:1 to pairs consisting of a homomorphism of rings $A \to k$ and a $k$-derivation $A \to k$. This is the observation on which everything else rests. $\endgroup$
    – Martin Brandenburg
    Mar 26, 2013 at 9:51
  • $\begingroup$ @MikeBattaglia There is a related explanation of first order infinitesmial symmetries in one of Qiaochu Yuan's old blogposts that may or may not help you develop a picture of the above idea. $\endgroup$
    – rschwieb
    Mar 26, 2013 at 10:01
  • $\begingroup$ Thanks for the references, everyone - will check those out. $\endgroup$
    – Mike Battaglia
    Mar 27, 2013 at 21:20
6
$\begingroup$

I think no one mentioned Synthetic differential geometry, there you have nonzero quantities with $dx^2=0$. For a very readable introduction I suggest:

Bell, A primer of infinitesimal Analysis

$\endgroup$
6
  • 6
    $\begingroup$ Note that these quantities aren't actually nonzero. What one can say about them is that one cannot prove that they are zero; nor can one prove that they are nonzero. This is of course only possible when the background logic is intuitionistic. $\endgroup$
    – Mikhail Katz
    Nov 28, 2013 at 19:14
  • $\begingroup$ Right. But at least one can prove that the set of all nilsquare elements does not reduce to the set $\{0\}$ $\endgroup$
    – Michael Bächtold
    Nov 28, 2013 at 21:44
  • 1
    $\begingroup$ How does that go? Is this really a "set" in the traditional sense? I recall that one needs a more sophisticated topos theory setting to make this work. Does one get a nonempty set of which one cannot exhibit any element? $\endgroup$
    – Mikhail Katz
    Nov 29, 2013 at 8:22
  • 1
    $\begingroup$ @user72694 have a look at page 5 of these notes: home.sandiego.edu/~shulman/papers/sdg-pizza-seminar.pdf $\endgroup$
    – Michael Bächtold
    Nov 29, 2013 at 17:39
  • $\begingroup$ @user72694, just curious, why do you need intuitionistic logic to have it so you can't prove the quantities are zero, nor can you prove they're nonzero? For instance, in ZFC, you can't prove that $|\omega_1| = |\Bbb R|$, nor can you prove that $|\omega_1| \neq |\Bbb R|$, and the background logic is classical. $\endgroup$
    – Mike Battaglia
    Nov 24, 2015 at 20:06
5
$\begingroup$

The answers by Ittay Weiss and Martin Brandenburg are helpful. I would like to point out a more direct shorcoming of the dual numbers as far as analysis (and even calculus) is concerned is that it is not clear how to extend a generic real function to the dual numbers, even say a $C^\infty$ smooth function. Thus, if one wishes to form a ratio of infinitesimals involved in the definition of the derivative, it is not clear what should appear in the numerator. Over the hyperreals, one has a systematic way of extending every real function to the wider hyperreal domain, and the transfer principle (which is arguably a formalisation of the Leibnizian Law of Continuity) ensures that such an extension is meaningful.

For this reason, the answer to the original question would be: No, dual numbers are insufficient to capture "Newton's historical method for doing calculus". The hyperreals provide a framework where the procedures of 17th century infinitesimal calculus can be successfully formalized.

$\endgroup$
4
  • $\begingroup$ I never saw this before, but since I'm seeing it now: I mentioned above, in my comments to Ittay, an algebraic structure where $\omega = 1/\epsilon$ is defined, with the property $\omega^2 = \infty$ and $-\omega^2 = -\infty$, taken from the extended real line $\mathbb{R} \cup \{\infty, -\infty\}$. Thus, every number in this system is of the form $a+b\epsilon+c\omega$, other than the special numbers $\infty$ and $-\infty$. This makes all sorts of things invertible which weren't invertible before. Would this still pose a problem in extending generic real functions to the dual numbers? $\endgroup$
    – Mike Battaglia
    Aug 16, 2013 at 7:01
  • 3
    $\begingroup$ @Mike: One immediate problem is that the condition $\epsilon^2=0$ would not allow us to carry out the usual algebraic simplifications on the relation $(\epsilon\omega)^2=1$. Thus one of the ordinary rules of algebra has to break down, which is inconvenient in a number system. In the hyperreal framework all ordinary rules of algebra continue to hold over the extended domain. $\endgroup$
    – Mikhail Katz
    Aug 16, 2013 at 7:49
  • $\begingroup$ Aha, great point. I never saw that before. Thanks. $\endgroup$
    – Mike Battaglia
    Aug 16, 2013 at 8:02
  • $\begingroup$ Just thinking about this again after many years - it would seem any analytic function can be extended to the dual numbers, since you can just use the Taylor series and formally plug in, for instance $1+\epsilon$ or whatever. Then using this and the ordering on the dual numbers, you can get the same thing for piecewise-analytic functions. But it isn't quite the same as nonstandard analysis in that, for instance, if you try $1/(1-(1+\epsilon))$ you get $1 + (1+\epsilon) + (1+\epsilon)^2 + ... = 1 + (1+\epsilon) + (1+2\epsilon) + (1+3\epsilon)$ and thus it doesn't converge. $\endgroup$
    – Mike Battaglia
    Mar 24, 2022 at 21:03
5
$\begingroup$

In the dual numbers for any differentiable function holds $ f(x+\epsilon) = f(x) + \epsilon f^\prime (x)$. This is enough to handle computationally 1st derivatives. Of course it is not enough for the conventional definition of second derivative. So you can consider the duals as a computational model. Of course, on of the drawbacks is that the purely imaginary numbers are not invertible.

$\endgroup$
12
  • 1
    $\begingroup$ This is a dubious claim. How do you define $f(x+\epsilon)$ for dual numbers if $f$ is an ordinary function? $\endgroup$
    – Mikhail Katz
    Dec 19, 2014 at 8:59
  • 1
    $\begingroup$ Well you can extend the function to the duals. On the real axis of the dual plane it will retain the same values as the function prototype acting on the reals. Automatic differentiation algorithms rely exactly on this extension. $\endgroup$
    – user48672
    Dec 20, 2014 at 0:28
  • 2
    $\begingroup$ You can define it that way but this definition is useless for defining the derivative of $f$, because according to your definition the derivative $\frac{f(x+dx)}{dx}$ will be always zero, with $dx$ the nilpotent infinitesimal. $\endgroup$
    – Mikhail Katz
    Dec 21, 2014 at 9:35
  • 2
    $\begingroup$ you missed the point. As in Smooth Infinitesimal Analysis, one could define the derivative $L$ by $f(x+\epsilon)= f(x)+L\epsilon$. However, unlike Smooth Infinitesimal Analysis, there does not seem to be any useful way of defining $f$ at $x+\epsilon$ in the dual numbers. $\endgroup$
    – Mikhail Katz
    Dec 22, 2014 at 9:37
  • 2
    $\begingroup$ That would be nice and Terry Tao has some ideas in this direction (he has a page on his blog called "cheap version of nonstandard analysis" or something like that). The problem is that as far as I can see the approach with the dual numbers does not work. I like the idea of finding cheap versions but Tao's is still considerably more involved than dual numbers. Again the difference is that Tao's approach works. $\endgroup$
    – Mikhail Katz
    Dec 23, 2014 at 9:15
4
$\begingroup$

Duals Numbers, attributed to Eduard Study, are already practically used, for example here:
https://github.com/JuliaDiff/DualNumbers.jl

From my point of view division seems not so much a problem, since it fails when ordinary division would also fail. Here is a set of arithmetic operations defined for duals, I am writing (x,y) instead of x+εy:

$$-(x,y) = (-x,-y)$$

$$(x,y)+(z,t) = (x+z,y+t)$$

$$(x,y)*(z,t) = (x*z,x*t+y*z)$$

$$\frac{(x,y)}{(z,t)} = (\frac{x}{z}, \frac{y*z-x*t}{z^2})$$

I guess the claim that duals cannot be used to define derivative, stems from a confusion with Jerome Keislers standard part. He writes translated to dual equations the following, and division is exactly the problem:

f((x,h)) - f((x,0))
------------------- = (f'(x), e) /* doesn't work */
      (0,h)

But if we use the hypothesis:

f((x,y)) = (f(x), f'(x)*y)

We then find the following by using this hypothesis and the aforementioned arithmetic operations:

f((x,h)) - f((x,0))
------------------- = (0, f'(x))  /* works */
      (h,0)

And if this isn't convincing enough, we can also use the hypothesis to show, that duals reflect the chain rule:

f(g( (x,1) )) = f( (g(x), g'(x)) )

              = ( f(g(x)), f'(g(x))*g'(x) )

Hyperduals are an extension of duals where second or higher order derivatives can be also calculated.

But currently I rather would wish for duals that can compute f(x+) and f(x-) for me, i.e. left and right derivative. Currently experimenting.

$\endgroup$
3
$\begingroup$

Yes... and no....

On the one hand, the dual numbers $\mathbb{R}[\epsilon] / (\epsilon^2)$ are a topological ring, and the projection $\mathbb{R}[\epsilon] / (\epsilon^2) \to \mathbb{R}$ is a vector bundle over the real line. In fact, it is isomorphic (as a vector bundle) to the tangent bundle $T\mathbb{R}$ and to the cotangent bundle $T^*\mathbb{R}$ in a rather suggestive way.

On the other hand, aside from being a neat way to differentiate polynomials, I'm not sure it actually does anything for you. e.g. while it's interesting to organize derivative information such as $\log(x+\epsilon y) = \log(x) + \epsilon \frac{y}{x}$, I don't know if it actually does anything to help you derive such formula.

On the cotangent side, matters are worse — I'm not aware of anything the dual numbers could do for you that the exterior algebra doesn't do as well or better.

$\endgroup$
1
  • $\begingroup$ If you assume that your function is analytic then the result follows trivially. This is not very restrictive for applications because the special functions are analytic. $\endgroup$
    – user48672
    Jan 30, 2020 at 23:02
0
$\begingroup$

This seems to be generating a lot of confusion.

It is simply that f(x+he)=f(x)+he df/dx where ee=0 ; algebraically and precisley for Taylor expandable differentiable functions.(ie analytic functions)

You can go through the table of all the elementary functions and write out the answer using this algebraic definition. Also proove all properties of derivatives. It is no better nor worse than the Lagrange definition of derivative, of whatever is sitting on the second term of the expansion.

Probably good for educational purposes, no torture using limit concept, the derivative is simply there. Of course analyists may misunderstand the algebra part. It is a ring, not a field. The he part is an ideal of the ring.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .