Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Can anyone please help me explain this fact? I tried to read some articles on the web about floating point but it is always a hard topic for me to understand.

This is what I get from Python 3.3.0

enter image description here

A brief to medium explanation is enough. It is really appreciated. Thanks!

share|cite|improve this question
3  
Here's a nice video that explains how these floating point errors come about. Long story short: it's because of truncated "repeating decimals" when numbers are expressed in binary. – Omnomnomnom Mar 13 at 23:11
3  
Computers have to do lots of weird stuff internally to even deal with floating point arithmetic, and they can never do it perfectly (there will always be some amount of error for non-terminating decimals). The differences you're seeing there are the product error in the particular methods being used. – Justin Benfield Mar 13 at 23:11
6  
Obligatory link to stackoverflow: Is floating point math broken? – LutzL Mar 13 at 23:26
2  
Why does 3*(1/3) = 0.99999999 in decimal? – immibis Mar 14 at 0:34
3  

The simple answer is because the computer stores numbers in binary: $0.3$ should have been stored as $$0.01001100110011001100\ldots$$However, a computer does not have infinite storage space, so it has to round it to some specified number of bits. Therefore it is not exactly $0.3$ that gets stored, but something roughly $10^{-16}$ smaller or larger (depending on what kind of float you're using). That rounding error is what you see in your calculations.

You can probably see that rounding error more directly by doing calculations like 0.1+0.2-0.3 or 0.1*2-0.2. Some of them will actually give 0, but a lot of them will give answers somewhere around the order of E-16 or E-17

Note that this is also why using == to compare floating point numbers is a bad idea. Numbers that mathematically should've been equal might be unequal inside the computer.

share|cite|improve this answer

Take some more digits to capture more precisely what the binary content of those floating point numbers actually is (even though only 16 digits are necessary to identify each of these numbers):

>>> print "%.50f" % 0.3
0.29999999999999998889776975374843459576368331909180
>>> print "%.50f" % (2*0.3)
0.59999999999999997779553950749686919152736663818359
>>> print "%.50f" % (3*0.3)
0.89999999999999991118215802998747676610946655273438
>>> print "%.50f" % 0.2
0.20000000000000001110223024625156540423631668090820
>>> print "%.50f" % (3*0.2)
0.60000000000000008881784197001252323389053344726562

So you see that there is no binary 0.2 or 0.3 since those are infinite periodic binary fractions.

share|cite|improve this answer

Consider what the individual digits of a number like 12.34 mean. Its value is $1\cdot10^1 + 2\cdot10^0 + 3\cdot10^{-1} + 4\cdot10^{-2}$. This is just what you learned in elementary school about the "ones place" and the "tens place," but written out in a more formal mathematical notation. Computers handle floating point numbers the same way, only they are far better at base 2 than they are at base 10, so they do the exact same process in binary. 5.25 gets broken into $1\cdot2^1 + 0\cdot2^1 + 1*\cdot2^0 + 0\cdot2^{-1} + 1\cdot2^{-2}$. If I were to rewrite it in the normal decimal form, but use base 2 values instead of base 10, $5.25_{10}$ would be written as $101.01_2$ (The base is notated here with a subscript, which is a very common notation to use when you are working with different bases. It helps keep the numbers straight). This is how computers typically approach floating point numbers.

If I ask you what 1/3 is as a decimal, you'd say something like $0.33333..._{10}$ However, if you kept writing 3's, youd run out of space, so you'd probably say "it's close to $0.33333_{10}$." Computers do the same thing. While 1/2 in base2 is $0.1_2$ and 1/8 is $0.001_2$ in base 2, not all numbers work out so cleanly. In particular 3/10 ($0.3_{10}$ in decimal) is equal to $0.010011001100..._2$ which must be rounded. It is these rounding errors that are causing the unusual effects you see.

As a general rule, one should never rely on exact equivalence with == to determine if two floating point numbers are equal, due to these rounding errors. One always compares them with some tolerance.

Many CPUs have a "decimal" mode for their floating point numbers which use base 10 instead of base 2. This can be slower, or lower precision, but it is a popular solution in situations like banking, where exact handling of fractions of a penny have to match up exactly. Since it is natural for us to watch for rounding errors in decimal, this causes the numbers to behave more intuitively. However, this has generally gone by the wayside. Almost everyone uses the normal IEE-754 binary representation of floating point numbers, because it is a very well rounded format. Those who need exactness often use fixed point math instead, which guarantees exact results using integer arithmetic.

share|cite|improve this answer

Binary floating point arithmetic can only exactly represent fractions with a power of $2$ as denominator.

$0.3$ is not such a fraction, so it is represented by a number that is off by at most half the smallest unit of the floating point number. If you multiply this by $3$, the result may be off by as much as $0.5$ of the smallest unit for $0.3$ before rounding, and off by $0.75$ of the smallest unit for $0.9$ after rounding. Which means it may not be the same as what the computer would choose for $0.9$ when asked directly (since it then would try to be off at most by $0.5$).

And that's for an FPU that has the minimum possible errors.

share|cite|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.