0
$\begingroup$

Trying to understand why the exponential function e^x is sensitive to round-off error. Googled a ton, did not find a clear answer for me. Please recommend some readings/videos.

$\endgroup$
2
  • 2
    $\begingroup$ One reason is surely how fast the function changes: for a point $(x+h, e^{x+h})$, the instantaneous gradient is $\lim_{h \to 0} \frac{e^{x+h}-e^x}{h} = e^x$, so the error from rounding increases exponentially. $\endgroup$
    – Toby Mak
    Commented Oct 11, 2019 at 5:53
  • $\begingroup$ Thanks, this is helpful! $\endgroup$ Commented Oct 11, 2019 at 15:32

1 Answer 1

3
$\begingroup$

Imagine you're trying to calculate $e^x$ for some $x$, but you round off $x$ to some other number $y$ and compute $e^y$ instead. Sensitivity comes down to trying to understand the following question.

If your error in the input is small, is the error in your output also small?

In other words, if $y$ is close to $x$, does that mean $e^y$ is close to $e^x$?

In order to nail this down, we need to nail down a definition of "close". When talking about rounding errors, a natural thing to look at is relative error (percentagewise error). For example, suppose that you round up and $y$ ends up being $1$ percent larger than $x$. We can express this as $y=1.01x$, or as $\frac{y}{x}=1.01$. What sort of error does this lead to in the exponential? The ratio between our answer and the correct answer is $$\frac{e^y}{e^x}=\frac{e^{1.01 x}}{e^x}=e^{1.01x-0.01x}=e^{0.01x}.$$

And here we see the problem: If $x$ is at all large, this ratio can be much larger than $1.01$. Even if $x$ is, say, $10$, the error in the output is already about $10$ times larger than the $1$ percent error in our input. This magnification of error is what is being referred to by sensitivity.

$\endgroup$
3
  • $\begingroup$ many thanks, I wish books were as good as you are in your explanations! $\endgroup$ Commented Oct 11, 2019 at 15:32
  • $\begingroup$ one more q. How the derivative of exponential function e^x predict it's roundoff error? The derivative is the same, e^x. I am confused here. $\endgroup$ Commented Oct 12, 2019 at 20:56
  • 1
    $\begingroup$ The derivative is particularly relevant when you're considering additive error (measuring error by the difference between your answer and the correct answer instead of the ratio). The idea is that if we have a function $f(x)$ and instead of inputting $x$ we input $x+h$ for some small $h$, then the definition of derivative means we have $$f(x+h) \approx f(x) + h f'(x).$$ So the additive error $h$ got magnified by a factor of $f'(x)$, meaning you may have problems if $f'(x)$ can be large. $\endgroup$ Commented Oct 12, 2019 at 21:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .