Finding a numerical stable way to evaluate different functions

Question

For each of the following functions, suggest a numerical stable way to evaluate $y$ for small $x$ in order to avoid loss of precision:

$y=x^2-\sin(x)^2$ - ignore terms of order $o(x^7$)

$y=\frac{x^2-\sin(x)^2}{2-\sqrt{x+4}}$ - ignore terms of order $o(x^6$)

$y=2x+e^x-e^{3x}$ - ignore terms of order $o(x^3$)

$y=\log(\frac{1+x}{1-x})$ - ignore terms of order $o(x^3)=O(x^4$)

$y=\sqrt{e^{-\frac{2}{3}x}-e^{-x}}$ - ignore terms of order $o(x)$

I have never seen something like this before even though I have been taking a course in numerical analysis. I have only been learning how to implement numerical methods. I was never taught why it was necessary, but I am familiar with the floating point architecture, rounding error, loss of precision by subtraction, etc. because I looked it up myself in order to make sense of why I had to learn all that stuff. Therefore I am not sure how to solve this problem properly. However, because of the specifications about ignoring some terms, I think the questioner wants me to do Taylor expansions of one or more terms for each function. But what does little and big o mean in this context? For instance what does $o(x^3)=O(x^4$) mean? Does it mean that I have to Taylor expand to degree 2 and that the error term is of order 3? If this is true, what is then the meaning of $o(x)$? Does it mean that I just have to replace some term with a constant? How do I generally identify what the problem is for each function? I mean what is the problem with evaluating $y=x^2-\sin(x)^2$ for small $x$? Is it that $\sin(x)$ will take on small values that may not be represented perfectly by the floating point architecture so we will get a rounding error? I hope my questions make sense and I will be very grateful for a clarifying answer.

Answer 1

• Indeed, the first step would be to identify the problem, i.e., for which values of $x$ is the formula numerically unstable.

• Typically, the issue in these kinds of formulas is cancellation, i.e., subtracting two nearly-identical numbers.

• Here is a potential solution for the first question: For $x$ close to zero, $x^2$ and $\sin(x)^2$ are nearly the same, so subtracting one from the other will result in a huge loss of precision. We can rewrite the equation as \begin{align} x^2 - \sin(x)^2 &= (x+\sin(x))(x-\sin(x))\,. \end{align} Now the first factor is not a problem (because for $x\approx 0$, both $x$ and $\sin(x)$ have the same sign, thus no cancellation). But the second term is still problematic. This can indeed be solved with a Taylor expansion. \begin{align} \sin(x) &= x - x^3/3! + x^5/5! + O(x^7)... \\ x^2 - \sin(x)^2 &= (x + \sin(x))(x^3/3! - x^5/5!) + O(x^8) \end{align} This is already pretty good, but can be improved slightly more (because there might still be some cancellation in the minus sign). Simply factoring out common factors (special case of Horner's method) gives us \begin{align} x^2 - \sin(x)^2 &= x^3(x + \sin(x))(1/3! - x^2/5!) + O(x^8) \end{align} which is now totally fine for $x\approx 0$.