Proof of Taylor's series expansion with two terms I am looking for a simple direct proof of the fact that
$$
\frac{\frac{f(x + \Delta x) - f(x)}{\Delta x} -f'(x)}{\Delta x} \stackrel{\Delta x \to 0}{\to} \frac{1}{2}f''(x),
$$
or, equivalently,
$$
f(x+\Delta x) = f(x) + f'(x)\Delta x + \frac{1}{2}f''(x)\Delta x^2 + o(\Delta x^2).
$$
holds for a twice-differentiable $f(x)$.
I remember there were times when I could derive this directly from the following definition of a derivative:
$$
f(x+\Delta x) = f(x) + f'(x)\Delta x + o(\Delta x)
$$
in a couple of simple lines.
A long time has passed since then and now I need to either recollect this magical "obvious" proof of mine or find out I was wrong then and the actual proof is more involved.
 A: The limit (in what follows I let $h=\Delta x$ for sake of simplicity):
$$\lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h}$$
comes in the indeterminate $0/0$ form.
If you're allowed to apply de l'Hôpital's rule, you can write:
$$\begin{split} \lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h} &= \lim_{h\to 0} \frac{f(x+h)-f(x) - f^\prime (x)\ h}{h^2}\\
& \stackrel{H}{=} \lim_{h\to 0} \frac{f^\prime (x+h) -f^\prime (x)}{2h} \quad \text{(here you took derivatives w.r.t. } h \text{)}\\
& = \frac{1}{2}\ f^{\prime \prime} (x)
\end{split}$$
therefore:
$$f(x+h)=f(x)+f^\prime (x)\ h+\frac{1}{2}\ f^{\prime \prime} (x)\ h^2 +\text{o}(h^2)\; .$$
A: After some tinkering I've come up with a proof which is still not the one I am hoping to find (if there is one at all), but it is closer to what I need in the sense that it is obviously generalizable to the multidimensional case, it is reasonably short, and it (kind-of) only relies on the very basic definitions of the gradient/derivative.
We start by defining a function $g(\mathbf{h}): \mathbb{R}^n \to \mathbb{R}$ as follows:
$$
g(\mathbf{h}) = f(\mathbf{x} + \mathbf{h}) - f(\mathbf{x}) - \nabla f(\mathbf{x})^T\mathbf{h} - \frac{1}{2}\mathbf{h}^T\nabla^2f(\mathbf{x})\mathbf{h}
$$
It now remains to show now that $g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert^2)$.
Note that $g(\mathbf{0}) = 0$, $\nabla g(\mathbf{0}) = \mathbf{0}_n$ and $\nabla^2 g(\mathbf{0}) = \mathbf{0}_{n\times n}$. Then
$$
\frac{\nabla g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{\nabla g(\mathbf{h}) - \nabla g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla^2 g(\mathbf{0}) = \mathbf{0},
$$
and consequently $\nabla g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert)$. Next, repeating the same step for $g(\mathbf{h})$ we have that
$$
\frac{g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{g(\mathbf{h}) - g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla g(\mathbf{h}) = o(\Vert\mathbf{h}\Vert),
$$
which implies that $g(\mathbf{h})$ is indeed $o(\Vert \mathbf{h} \Vert^2)$. Substituting  $o(\Vert \mathbf{h} \Vert^2)$ instead of $g(\mathbf{h})$ into the first equation, we obtain the necessary result.
Now this is still not simple enough, to my mind, and there should be something better and "more obvious".
