2
$\begingroup$

I have tried this on my Windows 7 calculator with $\sqrt9 -3 $ it too gives some weird answer- ie$1.1546388020691628168216106791278e-37$. And so for any $n$(positive) $\sqrt n^2-n= wierd_ .answer$
Why does this happen?

$\endgroup$
3

2 Answers 2

3
$\begingroup$

Presumable, the Windows 7 calculator uses floating point arithmetic to do these computations.

The IEEE 754 floating point standard requires that basic arithmetic operations (which include sqrt) are correctly rounded: the result should be as if it was computed with infinite precision and then correctly rounded to fit in a double.

Apparently, the Windows 7 calculator has a bug: instead of the correct answer 2 (which is of course exactly representable in a double) it seems to come up with the largest double smaller than 2. If you then subtract 2, you get something very close to but not exactly equal to 0.

$\endgroup$
2
  • 1
    $\begingroup$ I don't think it is a bug, roots are calculated afaik via iterations so the errors may grow extremely. If you have double floating point precision, shouldn't the largest machine number smaller than 2 be something like $2-10^{-16}$ and not something like $2-10^{-39}$ ? $\endgroup$ Commented Oct 6, 2013 at 16:02
  • 1
    $\begingroup$ I understand why it happens, but it's still wrong - it does not meet the specification. The makers probably didn't care and don't consider it a bug, though, but there I'm guessing. As for the the $2 - 10^{-16}$ vs. $2 - 10^{-39}$, it's probably extended double instead of double (128 bits instead of 64), but I didn't check. $\endgroup$ Commented Oct 6, 2013 at 16:05
1
$\begingroup$

The weird answer $-8.16\ldots e-39$ stands for $-8.16\ldots \cdot 10^{-39}$. It happens because a calculator just calculates numerical, so the maximal precision is the so called machine precision, which is usally something about $10^{-16}$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .