What is the convention for rounding operations in floating-point arithmetic?

Let's suppose we are using 64 bit precision and want to sum the numbers $1$ and $2^{-53}$ in floating point arithmetic. That is, we want to perform the following operation:

$1 + 2^{-53}$

We consider their representation:

$$1 \ \ \ \ \ = 2^0 \ \ \ \ \times 1. \underbrace{0000 \dots 0000}_\text{52 digits} $$

$$2^{-53}=2^{-53} \times 1.\underbrace{0000 \dots 0000}_\text{52 digits}$$

Then a shift is performed in order to sum, so we sum like:

$$ 2^0 \times 1. \underbrace{0000 \dots 0000}_\text{52 digits} $$

$$ + \ \ \ 2^{0} \times 0.\underbrace{0000 \dots 0000}_\text{52 digits}1 \ \ \ \ $$

Now the result is $$2^{0} \times 1.\underbrace{0000 \dots 00001}_\text{53 digits}$$ so a rounding must be performed. Both rounding up and rounding down produces the same rounding error, so what is actually done? I have tried this with Matlab and it rounds down to $2^{0} \times 1.\underbrace{0000 \dots 0000}_\text{52 digits}$.

How can I know how the machine rounds the number in any situation?

  • $\begingroup$ It's described on Wikipedia pretty well: en.wikipedia.org/wiki/Floating_point?wprov=sfsi1 $\endgroup$ – user251257 Nov 17 '16 at 12:05
  • $\begingroup$ I don't remember some of the specifics, but if I remember correctly, the operation is initially carried out in a way which returns more digits than will actually be returned, and then afterward it is rounded using a rounding rule. Knowing those additional bits allows you to apply "round to nearest, ties to even", which is the default rounding method. en.wikipedia.org/wiki/IEEE_floating_point#Rounding_rules $\endgroup$ – Ian Nov 17 '16 at 12:05
  • $\begingroup$ So $1+2^{-53}$ first looks like $1$, $52$ significant zeros, and then an insignificant $1$ (if I recall the exact memory layout correctly). Thus there is a tie between rounding it to $1+2^{-52}$ and rounding it to $1$ (each commits a rounding error of $2^{-53}$). Round to even decides to round it to $1$ specifically. $\endgroup$ – Ian Nov 17 '16 at 12:11

I guess you assume IEEE format (otherwise you are really lost without the detailed system spec). Two quotes from IEEE Std 754-2008:

  1. "Rounding takes a number regarded as infinitely precise and, if necessary, modifies it to fit in the destination’s format while signaling the inexact exception, underflow, or overflow when appropriate."

  2. "An implementation of this standard shall provide roundTiesToEven and the three directed rounding attributes."

In order to know the rounding environment you have to query the rounding mode and the precision from the floating point unit, normally there are library functions for this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.