I have tried this on my Windows 7 calculator with $\sqrt9 -3 $ it too gives some weird answer- ie$1.1546388020691628168216106791278e-37$. And so for any $n$(positive) $\sqrt n^2-n= wierd_ .answer$
Why does this happen?
$\begingroup$
$\endgroup$
3
asked Oct 6, 2013 at 15:42
-
2$\begingroup$ See stackoverflow.com/q/4428215/1090302 and en.wikipedia.org/wiki/Floating_point $\endgroup$– Zev ChonolesOct 6, 2013 at 15:44
-
$\begingroup$ Floats only store up to 8 bits of data iirc $\endgroup$– Don LarynxOct 6, 2013 at 15:46
-
$\begingroup$ So your calculator is close, but no cigar. Thus use a more reliant calculator. $\endgroup$– Michael HoppeOct 6, 2013 at 15:47
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
Presumable, the Windows 7 calculator uses floating point arithmetic to do these computations.
The IEEE 754 floating point standard requires that basic arithmetic operations (which include sqrt) are correctly rounded: the result should be as if it was computed with infinite precision and then correctly rounded to fit in a double.
Apparently, the Windows 7 calculator has a bug: instead of the correct answer 2 (which is of course exactly representable in a double) it seems to come up with the largest double smaller than 2. If you then subtract 2, you get something very close to but not exactly equal to 0.
answered Oct 6, 2013 at 15:56
-
1$\begingroup$ I don't think it is a bug, roots are calculated afaik via iterations so the errors may grow extremely. If you have double floating point precision, shouldn't the largest machine number smaller than 2 be something like $2-10^{-16}$ and not something like $2-10^{-39}$ ? $\endgroup$ Oct 6, 2013 at 16:02
-
1$\begingroup$ I understand why it happens, but it's still wrong - it does not meet the specification. The makers probably didn't care and don't consider it a bug, though, but there I'm guessing. As for the the $2 - 10^{-16}$ vs. $2 - 10^{-39}$, it's probably extended double instead of double (128 bits instead of 64), but I didn't check. $\endgroup$ Oct 6, 2013 at 16:05
$\begingroup$
$\endgroup$
The weird answer $-8.16\ldots e-39$ stands for $-8.16\ldots \cdot 10^{-39}$. It happens because a calculator just calculates numerical, so the maximal precision is the so called machine precision, which is usally something about $10^{-16}$
answered Oct 6, 2013 at 15:56