Formula derivation of the Second derivative I'm trying to prove the below equation:
The given function $f$ is twice differentiable. And the given formula is

$f''(x) = 2\lim_{h \rightarrow 0} \frac{f(x+h) -f(x) - f'(x)h } { h^2}$

I know this can be proved by l'Hospital's rule.

Since the numerator and denominator goes to zero as $h \rightarrow 0$, the given formula becomes  $lim_{h \rightarrow 0} \frac{f'(x+h)-f'(x)} {h}$, which equals to $f''(x)$.

But the reason I write this post is this:
When I tried to change the $f'(x)$ of the numerator, by definition of the derivative, the equation no longer true.
The trial is
$f''(x)$ = $2\lim_{h \rightarrow 0} \frac{f(x+h) -f(x) - f'(x)h } { h^2}$ = $2\lim_{h \rightarrow 0} \frac{f(x+h) -f(x) - h\cdot lim_{h \rightarrow 0} \frac{f(x) - f(x-h)} {h} } { h^2}$ = $2\lim_{h \rightarrow 0} \frac{f(x+h) -2f(x) + f(x-h) } { h^2}$ = $2f''(x)$.
But $f''(x)$ $=$ $2f''(x)$ can't be true for $f''(x)$ isn't zero.
The plausible inference for the obtaining the false result is the "change of $f'(x)$ to limit", from the second to the third of the above equalities, is wrong.
I think the possible explanation is, when the derivative change to the limit, the limit variable doesn't necessarily be $h$. The choice  of the limit variable doesn't affect the derivative of the $f$, for example, $f'(x) = \lim_{k \rightarrow 0} \frac{f(x+k) -f(x) } {k}$ is also okay. So the formula can't be combined by $lim_{h \rightarrow 0}$.
Any help would be appreciated.
 A: Note that Taylor's theorem tells us that
$$
f(x + h) = f(x) + f'(x) h + \frac 12 f''(x) h^2 + o(h^2).
$$
Using a $k$ for the inner limit provides us with some insight. In particular, we have
$$
\lim_{h \to 0}\frac{f(x + h) - f(x) - f'(x) h}{h^2} = 
\lim_{h \to 0}\frac{f(x + h) - f(x) - h\lim_{k \to 0}\frac{f(x) - f(x-k)}{k} }{h^2}\\
= 
\lim_{h \to 0}\lim_{k \to 0} \frac{f(x+h) - f(x) - \frac hk f(x) + \frac hk f(x-k)}{h^2}.
$$
Here, $\lim_{h \to 0}\lim_{k \to 0}$ indicates that we take a limit first with respect to $k$, then with respect to $h$.
Note that taking $k$ to be a function of $h$ and computing a limit over $h$ produces a different result. For your example, we see that taking $k(h) = h$ yields a limit of $2f''(x)$. More generally, taking $k(h) = h/a$ for $a \neq 0$ yields
$$
\lim_{h \to 0} \frac{f(x+h) - f(x) - \frac h{k(h)} f(x) + \frac h{k(h)} f(x-k(h))}{h^2}\\
= \lim_{h \to 0}
\frac{f(x+h) - f(x) - af(x) + a f(x-h/a)}{h^2}\\
= \lim_{h \to 0} \frac{f(x+h) - (1 + a)f(x)+ a f(x-h/a)}{h^2}\\
= \lim_{h \to 0} \frac 1{h^2}[f(x) + hf'(x) + \frac 12f''(x)h^2 - (1 + a)f(x)\\
\qquad + a (f(x) - f'(x) (h/a) + \frac 12 f''(x) (h/a)^2) + o(h^2)]\\
= \lim_{h \to 0} \frac{\frac 12(1 + 1/a)f''(x)h^2 + o(h^2)}{h^2}
\\ = \frac{a + 1}{2a} f''(x).
$$
