# Computers can't deal with limit of $\Delta x \to 0$

While I was studying about finite differences I came across an article that says "computers can't deal with limit of $\Delta x \to 0$ " in finite differences.But if computers can't deal with these equations does anybody know how they compute $\frac {d}{dx}$ of $x^2$ and other such equations.Or whether these equations are pre-written.

• After pew's answer, have a look at en.wikipedia.org/wiki/Automatic_differentiation. This would be interesting for you. Otherwise, numerical differentiation is very often used. – Claude Leibovici Nov 12 '14 at 9:56
• @ClaudeLeibovici:Are you saying that Automatic differentiation uses neither finite differences nor derivatives but rather use a number of pre-defined steps to compute derivative? – justin Nov 12 '14 at 10:08
• @justin the computers always follow some pre-defined steps. What these steps are depends on how you program it. You may use some very primitive definition and then use some set of axioms and definitions from which you derive all the derivatives including such primitive as of $x^2$. Another way would be to tabulate the most common functions and their derivatives and use several rules like chain rule, derivative of composition etc. and get the derivatives of more complex functions this way. – Ruslan Nov 12 '14 at 10:21
• @justin. Not at all ! In this kind of system, you enter the code for a function and it returns to you the analytical derivative. Have a look at www-sop.inria.fr/tropics/ad/whatisad.html. Their software Tapenade is on-line for free. – Claude Leibovici Nov 12 '14 at 10:26
• @ClaudeLeibovici:could you explain 'analytical derivative' or whether it simply means derivative. – justin Nov 12 '14 at 10:31

A table of the derivatives of primitive functions combined with differentiation rules yields an algorithm that allows a computer program to symbolically compute the derivative of any function that is a compound of primitive functions without having to rely on the limit definition at all (computer algebra).

In fact, unlike symbolic integration, this algorithm is fairly easy to implement.

Edit: It can only be re-emphasised that symbolic integration is a different matter entirely. In fact, while every function that is comprised of elementary functions has an elementary derivative, the same is not true for the antiderivative. This is a consequence of Liouville's theorem and finds application in the Risch "algorithm" (which is not an algorithm in the strict sense).

• whether computer solve ODE using differential equations or whether it solves using finite differences? – justin Nov 12 '14 at 9:53
• @justin: That depends on the system being used and is a very complex question. Some systems use a hybrid approach even when numerically solving ODEs, i.e. they first transform the equation symbolically (e.g for better stability properties) and then use Runge-Kutta or similar to generate a numeric solution. – user139000 Nov 12 '14 at 9:56
• :Whether the algorithm you mentioned uses finite differences to compute derivative?Because I couldn't understand what you meant by "unlike symbolic integration". – justin Nov 12 '14 at 9:59
• @justin: Symbolic differentiation algorithms do not use finite differences. I added some more information about symbolic integration before reading your comment, but in practice (industry application etc.), ODEs are almost always solved numerically only, for example using Runge-Kutta-like methods which proceed in "small" (but not infinitesimal) steps. Such methods will produce a numerical trajectory of the integral given a fixed initial value but not a symbolic representation ("formula") of the antiderivative. – user139000 Nov 12 '14 at 10:03
• :I couldn't understand the term 'symbolic differentiation'.I looked in wikipedia but it says about algorithms. – justin Nov 12 '14 at 11:03

This is not an answer but an illustration of automatic differentiation of a formula (I used Tapenade software online).

The source code I submitted is

  SUBROUTINE DUMMY(X,Y)
Y = X ** 2
END


which was interpreted as

  SUBROUTINE DUMMY(x, y)
IMPLICIT NONE
REAL x , y
y = x**2
END


and what I received is

  SUBROUTINE DUMMY_D(x, xd, y, yd)
IMPLICIT NONE
REAL x , xd , y , yd
yd = 2*x*xd
y = x**2
END


which now computes both the function $y$ and its derivative $\frac{dy}{dx}$ ($xd=x$).

• nice illustration.But could you show with derivative of $x^2$.I think that would be simple to understand much better. – justin Nov 12 '14 at 11:02
• that's good.But did they compute derivative of $x^2$ using just a predefined step i.e it's derivative would be $2x$ or using any formula? – justin Nov 12 '14 at 11:14
• They use the same rules as you do ! – Claude Leibovici Nov 12 '14 at 11:14
• okay.The reason why I asked this is whether it might be good if we use finite differences to compute a derivative or is there any place where finite differences are used to compute a derivative? – justin Nov 12 '14 at 11:19