Proof of Taylor's series expansion with two terms

I am looking for a simple direct proof of the fact that $$\frac{\frac{f(x + \Delta x) - f(x)}{\Delta x} -f'(x)}{\Delta x} \stackrel{\Delta x \to 0}{\to} \frac{1}{2}f''(x),$$ or, equivalently, $$f(x+\Delta x) = f(x) + f'(x)\Delta x + \frac{1}{2}f''(x)\Delta x^2 + o(\Delta x^2).$$

holds for a twice-differentiable $f(x)$.

I remember there were times when I could derive this directly from the following definition of a derivative: $$f(x+\Delta x) = f(x) + f'(x)\Delta x + o(\Delta x)$$ in a couple of simple lines.

A long time has passed since then and now I need to either recollect this magical "obvious" proof of mine or find out I was wrong then and the actual proof is more involved.

-

The limit (in what follows I let $h=\Delta x$ for sake of simplicity): $$\lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h}$$ comes in the indeterminate $0/0$ form.

If you're allowed to apply de l'Hôpital's rule, you can write: $$\begin{split} \lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h} &= \lim_{h\to 0} \frac{f(x+h)-f(x) - f^\prime (x)\ h}{h^2}\\ & \stackrel{H}{=} \lim_{h\to 0} \frac{f^\prime (x+h) -f^\prime (x)}{2h} \quad \text{(here you took derivatives w.r.t. } h \text{)}\\ & = \frac{1}{2}\ f^{\prime \prime} (x) \end{split}$$ therefore: $$f(x+h)=f(x)+f^\prime (x)\ h+\frac{1}{2}\ f^{\prime \prime} (x)\ h^2 +\text{o}(h^2)\; .$$

-
Thanks, this is correct indeed, but I would like to find a proof which does not use the l'Hospitals rule. I mean, something which would really only require the direct definition of the differentiation. The point is, I actually need to prove this result for a multidimensional case (but I hope that if I see the one-dimensional proof, I'll be able to easily generalize it). – KT. Mar 8 '12 at 17:47

After some tinkering I've come up with a proof which is still not the one I am hoping to find (if there is one at all), but it is closer to what I need in the sense that it is obviously generalizable to the multidimensional case, it is reasonably short, and it (kind-of) only relies on the very basic definitions of the gradient/derivative.

We start by defining a function $g(\mathbf{h}): \mathbb{R}^n \to \mathbb{R}$ as follows:

$$g(\mathbf{h}) = f(\mathbf{x} + \mathbf{h}) - f(\mathbf{x}) - \nabla f(\mathbf{x})^T\mathbf{h} - \frac{1}{2}\mathbf{h}^T\nabla^2f(\mathbf{x})\mathbf{h}$$

It now remains to show now that $g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert^2)$.

Note that $g(\mathbf{0}) = 0$, $\nabla g(\mathbf{0}) = \mathbf{0}_n$ and $\nabla^2 g(\mathbf{0}) = \mathbf{0}_{n\times n}$. Then $$\frac{\nabla g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{\nabla g(\mathbf{h}) - \nabla g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla^2 g(\mathbf{0}) = \mathbf{0},$$ and consequently $\nabla g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert)$. Next, repeating the same step for $g(\mathbf{h})$ we have that $$\frac{g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{g(\mathbf{h}) - g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla g(\mathbf{h}) = o(\Vert\mathbf{h}\Vert),$$ which implies that $g(\mathbf{h})$ is indeed $o(\Vert \mathbf{h} \Vert^2)$. Substituting $o(\Vert \mathbf{h} \Vert^2)$ instead of $g(\mathbf{h})$ into the first equation, we obtain the necessary result.

Now this is still not simple enough, to my mind, and there should be something better and "more obvious".

-