Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

As a programmer, we have been told about floating points errors on computer. Do Calculators have floating point error too?


0.1 (display) = .0999999998603016  (actual value used) on computers

Not really 0.1 But you can see it is close.

share|cite|improve this question
Every calculating machine that does inexact arithmetic will have floating point error... – J. M. Nov 21 '11 at 18:15
Calculators typically use decimal arithmetic and so can show "nicer" numbers than computers, which do binary arithmetic. – lhf Nov 21 '11 at 18:26
Also, calculators usually use more digits internally than they display. This tends to reduce roundoff error, but it does not eliminate it. – Robert Israel Nov 22 '11 at 0:19
"Calculators typically use decimal arithmetic" - more precisely, BCD. – J. M. Nov 22 '11 at 1:31
I dont think calculators have round off errors as in floating points. – TomCat Nov 22 '11 at 1:52

Calculators are computers, too; they're just smaller. Surely if we knew how to represent arbitrary real numbers inside calculators, we could do the same thing with desktop computers.

That said, it's possible—both on a calculator and on a computer—to represent some real numbers exactly. No computer I know of would represent $\frac12$ inexactly, since its binary expansion (0.1) is short enough to put inside a floating point register. More interestingly, you can also represent numbers like $\pi$ exactly, simply by storing them in symbolic form. In a nutshell, instead of trying to represent $\pi$ as a decimal (or binary) expansion, you just write down the symbol "$\pi$" (or, rather, whatever symbol the computer program uses for $\pi$).

share|cite|improve this answer
Calculators are, however, unlikely to represent $\pi$ symbolically. – Henning Makholm Nov 21 '11 at 18:23
If they did we would all be doomed. No body would use calculators in financial firms. I am sure they use different algorithm. But this is my instinct though. – TomCat Nov 21 '11 at 18:31

They do as long as you don't use one that uses symbolical calculations like the modern TI or Casio calculators.

One way to see it is to calculate iteratively roots of a number and then square the results again, if you do it often enough you will get a different result than the input due to numerical errors, this doesn't only have to do with the floating point representation but also with inaccurate root algorithms used in the calculator.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.