I came up with this way to estimate square roots by hand, but part of it doesn't seem to make sense.

Consider how $f(n) = \sqrt{n^2+\varepsilon} \approx n$ when $\varepsilon$ is small. Therefore, using the tangent line with slope $f'(n) = \frac{n}{\sqrt{n^2+\varepsilon}}$ to approximate $f$ gives

$f(n) \approx n +\varepsilon\cdot\frac{n}{\sqrt{n^2+\varepsilon}}$

However, if the original approximation is substituted in the denominator, this gives

$f(n) \approx n + \frac{\varepsilon n}{n} = n + \varepsilon$

Which obviously makes no sense. However, if the chain rule is skipped while taking the derivative, the approximation becomes

$f(n) \approx n + \frac{\varepsilon}{2n}$

Which is a good approximation. For example, when $\varepsilon = 1, \sqrt{n^2+1} \approx n+\frac{1}{2n},$ and when $\varepsilon = n, \sqrt{n^2+n} \approx n + \frac{1}{2}$

Why does this work only when the derivative is done incorrectly? I feel like I am missing something obvious, but I can't see why this works the way it does.

  • $\begingroup$ The derivative has a 1/2 term, by the chain rule. $\endgroup$ – gary Apr 5 '15 at 19:25
  • $\begingroup$ The derivative has 2n in the numerator, but also a 2 in the denominator, so they cancel. I skipped that step of work. EDIT: This comment was meant in response to an earlier comment that has since been deleted. $\endgroup$ – Apoapsis Apr 5 '15 at 19:25
  • $\begingroup$ Ah, yes, sorry. What I think may be an issue is that you have an $\epsilon$ both on top and a the bottom, which may create issues with cancellation. $\endgroup$ – gary Apr 5 '15 at 19:27

This seems like an improper application of the tangent line approximation. The usual approximation is $$f(x+\varepsilon)\approx f(x)+\varepsilon f'(x)\tag{*} $$ for $\varepsilon$ small. Your choice of $f$ doesn't match up with $(*)$. But using $f(x)=\sqrt x$ gives $$ \sqrt{x+\varepsilon}=f(x+\varepsilon)\approx\sqrt x+{\varepsilon\over{2\sqrt x}}$$ and plugging $x=n^2$ yields $$ \sqrt{n^2+\varepsilon}\approx n+\frac\varepsilon{2n},$$ the approximation you're seeking.


$\newcommand{\eps}{\varepsilon}$In order to make a first-order approximation, you must fix three pieces of data: A differentiable function $f$, a point $a$ of the domain where the approximation is taken, and a point $x$ at which $f(x)$ is approximated. In this situation, $$ f(x) \approx f(a) + f'(a) (x - a),\quad\text{or}\quad f(a + \eps) \approx f(a) + \eps \cdot f'(a). \tag{1} $$

In your set-up, you write "$f(n) = \sqrt{n^{2} + \eps}$". This is already unusual, in that $\eps$ is built into the definition of $f$.

Next, by writing "$f'(n) = n/\sqrt{n^{2} + \eps}$", you're taking $a = n$. Normally, first-order approximation is done in a way that $f'(a)$ is easy to evaluate exactly; here, $f'(a)$ involves a denominator to be approximated. That said, logically there's no problem.

In the third line, $$ f(n) \approx n + \eps \cdot \frac{n}{\sqrt{n^{2} + \eps}}, \tag{2} $$ each summand is problematic:

  • First, the left-hand side is $f(a)$, not $f(x)$.

  • Second, the "constant term" $n$ is not $f(a)$, i.e., $f(n) = \sqrt{n^{2} + \eps} \neq n$. Line (2) is not of the form (1).

  • Third, "$\eps$" plays two distinct roles: A number "baked into" the definition of $f$, and a small number representing $x - a$ in (1).

Here's what I suspect you were aiming for: define $g(x) = \sqrt{x^{2}}$ and $h(x) = \sqrt{x}$. You want the linear approximation of $g$ at $a = n$, and the approximation of $h$ at $a = n^{2}$: $$ g(n + \eps) \approx n + \eps \cdot \frac{n}{\sqrt{n^{2}}};\qquad h(n^{2} + \eps) \approx n + \eps \cdot \frac{1}{2\sqrt{n^{2}}}. $$ The first is exact (so long as $n$ and $n + \eps$ are positive) since $g(x) = x$ for $x > 0$. The second is the "chain rule skipped" approximation you were expecting.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.