# An Inconsistency in Numerical Approximation

Consider the expression

$$10^5 - \frac{10^{10}}{1+10^5}.$$

Using the elementary properties of fractions we can evaluate the expression as

$$10^5 - \frac{10^{10}}{1+10^5} = \frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = \frac{10^5}{1+10^5}\approx 1.$$

Note that the approximation $$10^5+1 \approx 10^5$$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get

$$10^5 - \frac{10^{10}}{1+10^5} \approx 10^5 - \frac{10^{10}}{10^5} = 0.$$

The same logic works for

$$10^p - \frac{10^{2p}}{1+10^p}$$

for arbitrary large $$p$$, so it cannot be simply an issue with the accuracy of the approximation.

Is there an easy explanation of what's going on here?

• en.wikipedia.org/wiki/Loss_of_significance might be a starting point Commented Nov 28, 2018 at 19:41
• This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example. Commented Nov 28, 2018 at 19:42
• relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$ Commented Nov 28, 2018 at 19:48
• @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– JMJ
Commented Nov 28, 2018 at 19:50
• Interestingly, the error compounds quickly; $$10^5 - \frac{10^{10}}{1+10^5} \approx 10^5 - \frac{10^{10}}{10^5} \approx 10^5 - \frac{10^{10}}{10^5-1}\approx\ldots \approx 10^5 - \frac{10^{10}}{2} \approx 10^5 - \frac{10^{10}}{1} \approx-10^{10}.$$ Commented Nov 29, 2018 at 1:17

It is simply an issue of accuracy of approximation. Let me write $$x = 10^p$$. Then your expression is $$x - \frac{x^2}{1+x}$$

Note that $$\frac{x^2}{1+x} = \frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$ so that $$x - \frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$

In your second calculation you only evaluated $$x^2/(1+x)$$ to within $$O(1)$$, not $$O(1/x)$$, so naturally you have an error at the end that is $$O(1)$$.

• Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-\frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– JMJ
Commented Nov 28, 2018 at 20:03

The first approximation is fine. The second on is not, because, $$10^5$$ and $$\dfrac{10^{10}}{1+10^5}$$ are large numbers with approximately the same size. You are saying that since $$10\,001$$ is close to $$10\,000$$, then $$1$$ is close to $$0$$.

When you approximate $$\frac{10^5}{1+10^5}=1-0.000099999000\cdots$$ with $$1$$, the error is on the order of $$10^{-5}$$.
But in the second case, the same error is multiplied by $$10^5$$, so that it is no more negligible.