Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am looking for a simple direct proof of the fact that $$ \frac{\frac{f(x + \Delta x) - f(x)}{\Delta x} -f'(x)}{\Delta x} \stackrel{\Delta x \to 0}{\to} \frac{1}{2}f''(x), $$ or, equivalently, $$ f(x+\Delta x) = f(x) + f'(x)\Delta x + \frac{1}{2}f''(x)\Delta x^2 + o(\Delta x^2). $$

holds for a twice-differentiable $f(x)$.

I remember there were times when I could derive this directly from the following definition of a derivative: $$ f(x+\Delta x) = f(x) + f'(x)\Delta x + o(\Delta x) $$ in a couple of simple lines.

A long time has passed since then and now I need to either recollect this magical "obvious" proof of mine or find out I was wrong then and the actual proof is more involved.

share|improve this question
add comment

2 Answers

After some tinkering I've come up with a proof which is still not the one I am hoping to find (if there is one at all), but it is closer to what I need in the sense that it is obviously generalizable to the multidimensional case, it is reasonably short, and it (kind-of) only relies on the very basic definitions of the gradient/derivative.

We start by defining a function $g(\mathbf{h}): \mathbb{R}^n \to \mathbb{R}$ as follows:

$$ g(\mathbf{h}) = f(\mathbf{x} + \mathbf{h}) - f(\mathbf{x}) - \nabla f(\mathbf{x})^T\mathbf{h} - \frac{1}{2}\mathbf{h}^T\nabla^2f(\mathbf{x})\mathbf{h} $$

It now remains to show now that $g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert^2)$.

Note that $g(\mathbf{0}) = 0$, $\nabla g(\mathbf{0}) = \mathbf{0}_n$ and $\nabla^2 g(\mathbf{0}) = \mathbf{0}_{n\times n}$. Then $$ \frac{\nabla g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{\nabla g(\mathbf{h}) - \nabla g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla^2 g(\mathbf{0}) = \mathbf{0}, $$ and consequently $\nabla g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert)$. Next, repeating the same step for $g(\mathbf{h})$ we have that $$ \frac{g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{g(\mathbf{h}) - g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla g(\mathbf{h}) = o(\Vert\mathbf{h}\Vert), $$ which implies that $g(\mathbf{h})$ is indeed $o(\Vert \mathbf{h} \Vert^2)$. Substituting $o(\Vert \mathbf{h} \Vert^2)$ instead of $g(\mathbf{h})$ into the first equation, we obtain the necessary result.

Now this is still not simple enough, to my mind, and there should be something better and "more obvious".

share|improve this answer
add comment

The limit (in what follows I let $h=\Delta x$ for sake of simplicity): $$\lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h}$$ comes in the indeterminate $0/0$ form.

If you're allowed to apply de l'Hôpital's rule, you can write: $$\begin{split} \lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h} &= \lim_{h\to 0} \frac{f(x+h)-f(x) - f^\prime (x)\ h}{h^2}\\ & \stackrel{H}{=} \lim_{h\to 0} \frac{f^\prime (x+h) -f^\prime (x)}{2h} \quad \text{(here you took derivatives w.r.t. } h \text{)}\\ & = \frac{1}{2}\ f^{\prime \prime} (x) \end{split}$$ therefore: $$f(x+h)=f(x)+f^\prime (x)\ h+\frac{1}{2}\ f^{\prime \prime} (x)\ h^2 +\text{o}(h^2)\; .$$

share|improve this answer
2  
Thanks, this is correct indeed, but I would like to find a proof which does not use the l'Hospitals rule. I mean, something which would really only require the direct definition of the differentiation. The point is, I actually need to prove this result for a multidimensional case (but I hope that if I see the one-dimensional proof, I'll be able to easily generalize it). –  KT. Mar 8 '12 at 17:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.