Intelligent Scalar Value for inputs to the Sigmoid Function I am currently coding a program that calls for passing an input value through the sigmoid function, though unfortunately, the software does not accommodate the large number exponentiation that the sigmoid function can entail (ie. -700 as an input would throw an error). Dividing the input by a scalar value (-700/10 for -70) seems the natural way around this, but I don’t want to lose precision. Is there a way to find an ideal, one-size fits all (or most) scalar divisor that minimizes the loss of precision that I can write into the program?
 A: If we are using $S(t)=\frac{1}{1+e^{-t}}$ as the sigmoid function, dividing the exponent $-t$ by some fixed scalar doesn't just have a loss in precision; it yields an incorrect answer that hopefully isn't too wrong. In the case of a large exponent like $-700$, the error is negligible. For an input like $-1$, the error is probably not negligible, depending on your application.
In the context of programming, you have an easier option. Any large enough value of $t$ should evaluate to $1$ in the floating point representation your chosen programming language is using, and any small enough value should evaluate to $0$. Assuming you have around $16$ digits of precision, you would want to solve $\left|1-\frac{1}{1+e^{-t}}\right|<10^{-16}$, which is almost equivalent to $t>16\ln10\approx36.84$.
You could do a similar calculation on the left hand side trying to get $|0-S(t)|<10^{-16}$ and similarly find $t<-16\ln10$. This will be symmetric.
In the context of programming, you often know instead that you have, for example, $53$ bits of accuracy. You would then write $t>53\ln2$ instead of $t>16\ln10$.
As far as what you actually do with those numbers, simply have an if/else structure in your code. For example, in python we could naively write
from math import exp

def sigmoid(t):
    if t > 36:
        return 1.
    elif t < -36:
        return 0.
    return 1/(1+exp(-t))

This would fix any "large exponent" problems you were having.
