Second derivative statement I have to prove the following:
$$f''(x)=\lim_{h\rightarrow0} \frac{f(x+h)+f(x-h)-2f(x)}{h^2}$$
I tried to just apply the definition of derivative, namely:
$$f'(x)=\lim_{h\rightarrow0} \frac{f(x+h)-f(x)}{h}$$
Then this:
$$f''(x)=\lim_{h\rightarrow0} \frac{f'(x+h)-f'(x)}{h}$$
$$f''(x)=\lim_{h\rightarrow0} \frac{\lim_{h\rightarrow0} \frac{f(x+2h)-f(x+h)}{h}-\lim_{h\rightarrow0} \frac{f(x+h)-f(x)}{h}}{h}$$
$$f''(x)=\lim_{h\rightarrow0}\frac{f(x+2h)-2f(x+h)+f(x)}{h^2}$$
But what now? I came to the conclusion that this is true for $f''(x-h)$, but not for $f''(x)$. How can I solve this problem?
 A: Now first of all your proof is wrong because you are using the same h for both limits which lead to the merge of the limits and got you $h^2$ in the denominator. The right way to write is as shown below:
\begin{align}
f''(x)&=\lim_{h\to 0}\frac{f'(x)-f'(x-h)}{h}\\[10px]
&=
\lim_{h\to 0}
\dfrac{
\lim\limits_{k\to0}\dfrac{f(x+k)-f(x)}{k}
-
\lim\limits_{k\to0}\dfrac{f(x-h+k)-f(x-h)}{k}
}
{
h
}
\end{align}
Now at this point I'm not sure if you would be able to continue.
On the other hand, this can be solved by using L'Hopital's Rule twice:
$$\lim\limits_{h\to 0}\dfrac{f(x+h)-2f(x)+f(x-h)}{h^2}\\=\lim\limits_{h\to 0}\dfrac{f'(x+h)-f'(x-h)}{2h}\\=\lim\limits_{h\to 0}\dfrac{f''(x+h)+f''(x-h)}{2}\\=f''(x)$$ 
A: This can be solved using Taylor expansion. 
$f(x+h)=f(x)+f'(x)h+\frac{f''(x)h^2}{2}+o(h^2)$
$f(x-h)=f(x)+f'(x)(-h)+\frac{f''(x)(-h)^2}{2}+o(h^2)$
Hence $f(x+h)+f(x-h)=2f(x)+f''(x)h^2+o(h^2)$. So:
$\frac{f(x+h)+f(x-h)-2f(x)}{h^2}=f''(x)+\frac{o(h^2)}{h^2}$ and it converges to $f''(x)$ when $h\to 0$. 
This solution allows us to assume that the second derivative exists only at the point $x$ itself and not in any neighborhood of it. 
A: You could do it directly using Taylor series. Consider
$$y=Af(x+h)+B f(x)+C f(x-h)$$ and expand using
$$f(x+h)=f(x)+h f'(x)+\frac 12 h^2f''(x)+\frac 16h^3f'''(x)+\cdots$$
$$f(x-h)=f(x)-h f'(x)+\frac 12 h^2f''(x)-\frac 16h^3f'''(x)+\cdots$$
$$y=(A+B+C)f(x)+(A-C)hf'(x)+\frac 12(A+C) h^2f''(x)+\frac 16(A-C)h^3f'''(x)+\cdots$$ So,$C=A$ to cancel the first derivative and $B=-2A$ for the first term. Then
$$y=Af(x+h)-2A f(x)+Af(x-h)=A h^2 f''(x)+\cdots$$
Divide everything by $A$
