10
$\begingroup$

I was studying Numerical Analysis by K. Mukherjee; there he discussed Loss of Significant Figures by Subtraction, as followed:

In the subtraction of two approximate numbers, a serious type of error may be present when the numbers are nearly equal. ...

He showed how the number of significant figures got decreased after subtraction.

In order to dig more, I googled and came across the term catastrophic cancellation.

I did try to read the wiki article but it was full of terms especially floating point arithmetic which I am really not acquainted it; I did try to read the latter's wiki article but couldn't comprehend the definition:

The term floating point refers to the fact that a number's radix point (decimal point, or, more commonly in computers, binary point) can "float"; that is, it can be placed anywhere relative to the significant digits of the number.

Radix point can float? Unfortunately, I failed to visualise that.

However, my main question, even if there is an error due to the loss of the significant figures, why is this error "catastrophic"?

I found in a decade old page that

The relative error here is infinite.

But how could the loss of significant figures led to an infinite magnitude of relative error?

Could anyone please explain the reason behind the term "catastrophic" keeping in mind that I'm not acquainted with floating point arithmetic?

$\endgroup$
  • 4
    $\begingroup$ You are going to have to understand floating point arithmetic before any of these answers will make any sense, or indeed, before reading any book entitled Numerical Analysis. It isn't that hard, though. $\endgroup$ – Nate Eldredge Sep 9 '16 at 15:29
  • 1
    $\begingroup$ It is called catastrophic, since it leads to a serious error in the computations. So if you calculate for example a trajectory describe by a system of ordinary differential equations, the errors sum up and will lead to a very different (wrong) trajectory. But I support Nate Eldredge's comment. $\endgroup$ – TheGeekGreek Sep 9 '16 at 15:31
  • 6
    $\begingroup$ @NateEldredge Considering how many professional programmers don't understand floating point arithmetic, "It isn't that hard" may be a tad optimistic. $\endgroup$ – Daniel Fischer Sep 9 '16 at 15:32
  • $\begingroup$ Well, then @NateEldredge, thanks for the advise; the book I'm following doesn't even mention that; that's why. $\endgroup$ – user142971 Sep 9 '16 at 15:32
20
$\begingroup$

The root of the evil is that calculators work with a finite number of digits (well, we can't afford machines with infinite resources yet). When you want to represent real-life numbers, some can be huge, some can be tiny, and this raises an issue.

Floating-point:

If the decimal point is fixed, you can' represent them all, there can be so-called overflow or underflow; and for intermediate orders of magnitude, you lose significant digits.

For example, using the $5.5$ fixed-point representation, $127500.$ and $0.0000096695$ cannot be represented, and $\pi=3.14159$ just uses six significant digits, four positions are wasted.

The floating-point representation solves these problems by specifying the position of the decimal point, as in the scientific notation:

$127500.=1.275000000\cdot10^5, 0.0000096695=9.669500000\cdot10^{-6}, \pi=3.141592654\cdot10^0$.

Catastrophic cancellation:

(This concept is actually unrelated to floating-point.)

Assume you want to play with the approximation $\pi\approx355/113=3.14159292\cdots$.

If you want to take the sum with $\pi$, all goes well.

$$\frac{3.14159265+3.14159292}2=3.14159278.$$

But now if you want to take the difference,

$$\frac{3.14159265-3.14159292}2=-0.000000135,$$

only three significant decimals remain. This is a catastrophy, because you started with nine figures and end-up with just three, and this is irreversible ! (Floating-point will not come to the rescue, $-0.000000135=-1.35\cdot10^{-7}$ still has three significant digits.)

This phenomenon makes some problems very difficult to solve numerically.


In some cases, catastrophic cancellation can be avoided. For example, when solving the quadratic equation

$$x^2-1000.001x+1=0,$$

the classical formula says

$$x=\frac{1000.001\pm\sqrt{1000.001^2-4}}2=\frac{100.001\pm999.999}2.$$

Assuming you can only compute with five significant digits, using floating-point, you obtain

$$x=\frac{1000.00\pm999.99}2=999.95\text{ and }0.005,$$ where the second root is very very inaccurate.

You can get a much better estimate by using the fact that the product of the roots is $1$, and the second root evaluates to

$$\frac1{999.95}=0.0010005$$

$\endgroup$
  • $\begingroup$ Thanks, for the answer. I wanted to know why in that linked page in my post above, it was said, the relative error is infinite; could you please shed some light on this? $\endgroup$ – user142971 Sep 9 '16 at 15:58
  • $\begingroup$ Also, unless you intentionally used \cdot, you could use \times to render $-2.7 \times 10^{-7}\;.$ $\endgroup$ – user142971 Sep 9 '16 at 16:01
  • $\begingroup$ Or else never mind :) $\endgroup$ – user142971 Sep 9 '16 at 16:02
  • $\begingroup$ @MAFIA36790: a matter of taste, I find the $\times$ a little intrusive. Anyway I admit that there can be confusion between $.$ and $\cdot$ $\endgroup$ – Yves Daoust Sep 9 '16 at 16:07
  • 1
    $\begingroup$ @MAFIA36790: my numerical shows a relative error of $400\%$. You can find an example with $\infty\%$. $\endgroup$ – Yves Daoust Sep 9 '16 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy