Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
Can anyone please help me explain this fact? I tried to read some articles on the web about floating point but it is always a hard topic for me to understand.
This is what I get from Python 3.3.0
A brief to medium explanation is enough. It is really appreciated. Thanks!
The simple answer is because the computer stores numbers in binary: $0.3$ should have been stored as $$0.01001100110011001100\ldots$$However, a computer does not have infinite storage space, so it has to round it to some specified number of bits. Therefore it is not exactly $0.3$ that gets stored, but something roughly $10^{-16}$ smaller or larger (depending on what kind of float you're using). That rounding error is what you see in your calculations.
You can probably see that rounding error more directly by doing calculations like 0.1+0.2-0.3 or 0.1*2-0.2. Some of them will actually give 0, but a lot of them will give answers somewhere around the order of E-16 or E-17
Note that this is also why using == to compare floating point numbers is a bad idea. Numbers that mathematically should've been equal might be unequal inside the computer.
Take some more digits to capture more precisely what the binary content of those floating point numbers actually is (even though only 16 digits are necessary to identify each of these numbers):
>>> print "%.50f" % 0.3
0.29999999999999998889776975374843459576368331909180
>>> print "%.50f" % (2*0.3)
0.59999999999999997779553950749686919152736663818359
>>> print "%.50f" % (3*0.3)
0.89999999999999991118215802998747676610946655273438
>>> print "%.50f" % 0.2
0.20000000000000001110223024625156540423631668090820
>>> print "%.50f" % (3*0.2)
0.60000000000000008881784197001252323389053344726562
So you see that there is no binary 0.2 or 0.3 since those are infinite periodic binary fractions.
Consider what the individual digits of a number like 12.34 mean. Its value is $1\cdot10^1 + 2\cdot10^0 + 3\cdot10^{-1} + 4\cdot10^{-2}$. This is just what you learned in elementary school about the "ones place" and the "tens place," but written out in a more formal mathematical notation. Computers handle floating point numbers the same way, only they are far better at base 2 than they are at base 10, so they do the exact same process in binary. 5.25 gets broken into $1\cdot2^1 + 0\cdot2^1 + 1*\cdot2^0 + 0\cdot2^{-1} + 1\cdot2^{-2}$. If I were to rewrite it in the normal decimal form, but use base 2 values instead of base 10, $5.25_{10}$ would be written as $101.01_2$ (The base is notated here with a subscript, which is a very common notation to use when you are working with different bases. It helps keep the numbers straight). This is how computers typically approach floating point numbers.
If I ask you what 1/3 is as a decimal, you'd say something like $0.33333..._{10}$ However, if you kept writing 3's, youd run out of space, so you'd probably say "it's close to $0.33333_{10}$." Computers do the same thing. While 1/2 in base2 is $0.1_2$ and 1/8 is $0.001_2$ in base 2, not all numbers work out so cleanly. In particular 3/10 ($0.3_{10}$ in decimal) is equal to $0.010011001100..._2$ which must be rounded. It is these rounding errors that are causing the unusual effects you see.
As a general rule, one should never rely on exact equivalence with == to determine if two floating point numbers are equal, due to these rounding errors. One always compares them with some tolerance.
Many CPUs have a "decimal" mode for their floating point numbers which use base 10 instead of base 2. This can be slower, or lower precision, but it is a popular solution in situations like banking, where exact handling of fractions of a penny have to match up exactly. Since it is natural for us to watch for rounding errors in decimal, this causes the numbers to behave more intuitively. However, this has generally gone by the wayside. Almost everyone uses the normal IEE-754 binary representation of floating point numbers, because it is a very well rounded format. Those who need exactness often use fixed point math instead, which guarantees exact results using integer arithmetic.
Binary floating point arithmetic can only exactly represent fractions with a power of $2$ as denominator.
$0.3$ is not such a fraction, so it is represented by a number that is off by at most half the smallest unit of the floating point number. If you multiply this by $3$, the result may be off by as much as $0.5$ of the smallest unit for $0.3$ before rounding, and off by $0.75$ of the smallest unit for $0.9$ after rounding. Which means it may not be the same as what the computer would choose for $0.9$ when asked directly (since it then would try to be off at most by $0.5$).
And that's for an FPU that has the minimum possible errors.
