I tried to do a simple linear regression using the "Numworks" calculator and unless I add many more data points or add a point where x=0 it doesn't give me the correct equation.

Could someone explain why is the equation

y = 2*x -1.776356839e-15

instead of

y = 2*x + 0

Is it because of the linear regression algorithm they use.

If this is the case shouldn't it still work for such a simple equation ?

I found the c++ code the calculator is using for linear model https://github.com/numworks/epsilon/blob/master/apps/shared/linear_regression_store.cpp#L87

The code seem to do the following steps:

x = {1, 2, 3}
y = {2, 4, 6}
slope = Covariance[x, y]  / Variance[x]
intercept = Mean[y] - slope * Mean[x]

enter image description here enter image description here

  • 3
    $\begingroup$ it's just how floating point arithmetic works, unfortunately $\endgroup$
    – Andrew
    Commented Aug 11, 2023 at 3:40
  • $\begingroup$ I understand that because of float calculation dont have perfect precision. But other calculator seem to get the perfect answer. $\endgroup$
    – skyde
    Commented Aug 11, 2023 at 7:20

1 Answer 1


That term, 1.776356839e-15, is almost exactly $2^{-49}$. This suggests roundoff error.

If you can look at the statistics for the regression calculation ($\sum x, \sum y, \sum xy$), and compare the results to the exact result, perhaps you can see where the roundoff occurs.

  • 1
    $\begingroup$ right but other calculator Ex: Desmos are giving the exact result see: desmos.com/calculator/daeupqyfew How is this possible? $\endgroup$
    – skyde
    Commented Aug 11, 2023 at 3:55
  • 1
    $\begingroup$ You need to look at the formulas used. $\endgroup$ Commented Aug 11, 2023 at 4:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .