# significant figures and numerical cancellation

I haven't really paid attention to significant figures and precision, and bypassed the importance of it by just memorizing all the rules learned from high school, but I realized this bites me hard when I have to encounter significant figures in numerical analysis class. As far as my knowledge goes, the number of significant figures is just the number of important digits.

But then I came across Why is 'catastrophic cancellation' called so?, and I'm still not clear why loss of significant digits would be "catastrophic" in the calculation. Is it because when a number is represented in a floating point machine, which has the fixed number of precise bits, and those important bits get thrown away? I'd appreciate it if anyone makes a further insight into this, and also about significant digits.

Loss of digits mean loss of accuracy in general. But sometimes you do calculations that look good on paper, but are numerically pretty bad. In the best case, your program will throw an error. It can also be worse, where the algorithm does work, but the result does not reflect your expectations.

Here are three examples where cancellation of significant digits can be pretty bad.

Let's start with an artificial example. If you calculate $$y = \frac{1}{a-\tilde{a}},$$ with close values $a$ and $\tilde{a}$ and you can't present that small number using floating point numbers, your code will (at best) throw an error.

That problem here is pretty obvious, so you should see it immediately. But if your algorithms and programs get more complex, it will be hard to find these weak spots.

Another application that often is used in numerics, is the approximation of the derivative by difference quotients. For example the Finite Difference Method, which is very common in the industry, uses difference quotients to approximate derivatives in partial differential equations.

$$f'(x) = \frac{f(x)-f(x+h)}{h}.$$

Even if $f'(x)\neq0$, if you choose $h$ very small, the enumerator will be numerically zero.

The task is to approximate the value of $\pi$.

I don't want to adorn myself with borrowed plumes, by simply copy-translate-pasting the whole article. Here is a rough idea what it says.

As we know, the area of the unit circle is $$A_c = \pi r^2 = \pi.$$ And we can approximate that area by the area $a_n$ of the $n$-sided polygon that inscribes the circle. Everything we need to calculate the area, is the length $s_n$ of one side of that polygon.

Knowing that length $s_n$ we can also calculate the length of the side of the $2n$-sided polygon with: $$s_{2n} = \sqrt{2-^{\star}2\sqrt{1-\frac{s_n^2}{4}}}.$$

That gives us a nice algorithm to approximate $\pi$.

Using that formula, you can see that in the numbers of the first table on the wikipedia page.
The computation has been done using double precision, so one expects 15correct digits. However, the best approximation is reached around step n= 6.554e+04 with 7 correct digits.
The problem is the operation marked with $^{\star}$, as $2$ and $2\sqrt{1-\frac{s_n^2}{4}}$ will eventually get close.

They fix that problem by rewriting the formula which avoids the subtraction. That leads to the values of the second table. As you can see, now you get the expected 15 significant digits.

I hope that the last example can be understood in the way I wrote it. If not, let me know.

You can find the reasoning why subtraction $x-y$ leads to cancellation if $|x|\approx |y|$, in one of my answers to another question.

• nice explanation. Thanks a lot – Ted Sep 25 '17 at 0:18