Where did the linear approximation/linearization formula come from? I understand that it takes root in the point-slope form and slope intercept form of a linear equation, but I don't understand where they got the exact formula $$f(x) \approx f(x_0)+f'(x_0)(x-x_0)$$ Also, where did Newton's Method come from? Can someone please explain this, or rather the individual components of the equation, to me?


  • $\begingroup$ This follows from Taylor series. $\endgroup$ – user122283 Apr 20 '14 at 17:37

Consider the two points $(x_0, f(x_0))$ and $(x,f(x))$. Now suppose we wanted to find the slope between these two points. Using the slope formula, we know that this slope is: \begin{align*} m &= \frac{f(x) - f(x_0)}{x - x_0} \\ m(x - x_0) &= f(x) - f(x_0) \\ f(x_0) + m(x - x_0) &= f(x) \\ f(x) &= f(x_0) + m(x - x_0) \end{align*}

Now provided that $x$ and $x_0$ are "close enough", we can approximate the slope of the tangent line of $f$ at $x_0$ to be $m$. Hence, we obtain: $$ f(x) \approx f(x_0) + f'(x_0)(x - x_0) $$

  • $\begingroup$ This makes a lot of sense! Thank you! $\endgroup$ – OpieDopee Apr 20 '14 at 17:52
  • $\begingroup$ No problem, glad I could help! =] $\endgroup$ – Adriano Apr 20 '14 at 17:54

Contrary to Sanath Devalapurkar's answer, this is not really an instance of Taylor series so much as Taylor series are a generalization of this. There are two parts to linear approximation: the formula for the line, and the fact that it is an approximation to the function.

  • The formula for the line is, as you say, just the point-slope formula for the line with slope $f'(x_0)$ passing through the point $(x_0, f(x_0))$; i.e. it is a line through a point of the graph of $f$ whose slope is the derivative of $f$ there.

  • The fact that this is actually an approximation to $f$ is the definition of the derivative. To see this, let $L(x) = f(x_0) + f'(x_0) (x - x_0)$ be the function for the tangent line. Then the definition of $f'(x_0)$ is $$\lim_{x \to x_0} \frac{f(x) - L(x)}{x - x_0} = \lim_{x \to x_0} \frac{f(x) - f(x_0)}{x - x_0} - f'(x_0) = 0,$$ which says, intuitively, that when $x$ is close enough to $x_0$ that their difference $x - x_0$ is tiny, the difference $f(x) - L(x)$ is tiny even when compared to that. This is the sense in which $L(x)$ is a good approximation to $f(x)$.

For Newton's method, see the Wikipedia article, which has a nice animation.

  • $\begingroup$ Thanks for clarifying things! :) $\endgroup$ – OpieDopee Apr 20 '14 at 22:01

The Taylor series (see the link for a "technical" introduction) of $f(x)$ about $x_0$ approximates the value of $f(x)$ in some sufficiently small neighborhood of $x_0$: $$f(x)= \sum_{n=0} ^ {\infty} \frac {f^{(n)}(x_0)}{n!} \, (x-x_0)^{n}$$ Your above written formula is the first-order approximation. To see why Taylor series work, visit this page. Also, see this M.SE question.

  • $\begingroup$ So that is the Taylor Series of $f'(x_0)$? $\endgroup$ – OpieDopee Apr 20 '14 at 17:45
  • 1
    $\begingroup$ @Ethan No - $f(x)\approx f(x_0)+f^\prime(x_0)(x-x_0)$ is the first order approximation of $f(x)$ about $x_0$. To see this intuitionistically, draw a circle about any point $x_0\in\mathbb{R}^2$. $\endgroup$ – user122283 Apr 20 '14 at 17:46
  • $\begingroup$ Ah, okay. I understand this now. Thank you! $\endgroup$ – OpieDopee Apr 20 '14 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.