The question I'd like to ask is this:
If $f''(0)$ exists, does $f'$ exist in a neighborhood of $0$?
Of course, under the standard definition of $f''(0)$, we have already assumed that $f'$ exists in a neighborhood of $0$. So instead:
Question: Is there a standard way to define $f''(0)$ as a limit expression that does not include $f'$ in it, and if so, can we deduce from the fact that $f''(0)$ exists that $f'$ exists in a neighborhood of $0$?
Details
If I know what $f'(0)$ is, I can make $f''(0)$ be the constant (if it exists) such that $$ \lim\limits_{h \to 0} \frac{f(h) - \left[ f(0) + f'(0) h + \frac{1}{2} f''(0) h^2\right]}{h^2} = 0. $$ i.e., the Taylor polynomial approximates $f$ to second order. Then, I could just plug in for $f'(0)$ the expression $\frac{f(h) - f(0)}{h}$. But this doesn't work; everything cancels out. Is there a different standard way to define $f''$ without using $f'$?
I should probably put some more work into answering this myself, but first I wanted to see if this is a standard or well-known question.