6
$\begingroup$

Consider the expression

$$ 10^5 - \frac{10^{10}}{1+10^5}. $$

Using the elementary properties of fractions we can evaluate the expression as

$$ 10^5 - \frac{10^{10}}{1+10^5} = \frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = \frac{10^5}{1+10^5}\approx 1. $$

Note that the approximation $10^5+1 \approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get

$$ 10^5 - \frac{10^{10}}{1+10^5} \approx 10^5 - \frac{10^{10}}{10^5} = 0. $$

The same logic works for

$$ 10^p - \frac{10^{2p}}{1+10^p} $$

for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.

Is there an easy explanation of what's going on here?

$\endgroup$
8
  • 4
    $\begingroup$ en.wikipedia.org/wiki/Loss_of_significance might be a starting point $\endgroup$
    – Thomas
    Commented Nov 28, 2018 at 19:41
  • 5
    $\begingroup$ This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example. $\endgroup$ Commented Nov 28, 2018 at 19:42
  • $\begingroup$ relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$ $\endgroup$
    – Vasili
    Commented Nov 28, 2018 at 19:48
  • $\begingroup$ @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly! $\endgroup$
    – JMJ
    Commented Nov 28, 2018 at 19:50
  • 1
    $\begingroup$ Interestingly, the error compounds quickly; $$10^5 - \frac{10^{10}}{1+10^5} \approx 10^5 - \frac{10^{10}}{10^5} \approx 10^5 - \frac{10^{10}}{10^5-1}\approx\ldots \approx 10^5 - \frac{10^{10}}{2} \approx 10^5 - \frac{10^{10}}{1} \approx-10^{10}.$$ $\endgroup$
    – Servaes
    Commented Nov 29, 2018 at 1:17

3 Answers 3

6
$\begingroup$

It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is $$ x - \frac{x^2}{1+x}$$

Note that $$\frac{x^2}{1+x} = \frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$ so that $$ x - \frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$

In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.

$\endgroup$
1
  • $\begingroup$ Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-\frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series. $\endgroup$
    – JMJ
    Commented Nov 28, 2018 at 20:03
1
$\begingroup$

The first approximation is fine. The second on is not, because, $10^5$ and $\dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10\,001$ is close to $10\,000$, then $1$ is close to $0$.

$\endgroup$
1
$\begingroup$

There is no paradox.

When you approximate $$\frac{10^5}{1+10^5}=1-0.000099999000\cdots$$ with $1$, the error is on the order of $10^{-5}$.

But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.

$\endgroup$
1
  • $\begingroup$ I don't recall claiming there was a paradox, only a numerical issue. $\endgroup$
    – JMJ
    Commented Nov 28, 2018 at 20:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .