Is there a limit which characterizes twice differentiability? If $f$ is twice differentiable at $x=a$, then we have
$$
f''(a) = \lim_{h \to 0} \frac{f(a+h)-2f(a)+f(a-h) }{h^2}
$$
However there are functions which are not twice differentiable for which this limit exists (for example, the signum function).
Is there a limit definition for $f''(a)$ which exists iff $f$ is twice differentiable?
 A: Why not just iterate the definition of the derivative: we see that $f$ is differentiable if the limit 
$$f'(x) = \lim_{h\to 0} \frac{f(x+h) - f(x)}{h}$$ 
exists. Then $f$ is twice differentiable if the limit 
$$f''(x) = \lim_{k\to 0} \frac{f'(x+k) - f'(x)}{k}$$ 
exists. Plugging in the definition for the first derivative, we see $f$ is twice differentiable if the double limit 
\begin{align*} 
f''(x) 
&= \lim_{k\to 0} \frac{\lim_{h\to 0} \frac{f(x+k+h)-f(x+k)}{h} - \lim_{h\to 0} \frac{f(x+h) - f(x)}{h}}{k}\\  
&= \lim_{k\to 0} \lim_{h\to 0}\frac{f(x+k+h)-f(x+k)-f(x+h)+f(x)}{hk}
\end{align*} 
exists. Of course, if this limit exists, this would imply that 
$$f''(x) = \lim_{h\to 0} \frac{f(x+h) - 2f(x) + f(x-h)}{h^2}$$
I'm not sure that we can use the latter as a definition since it would be derived from taking a specific value for $k$; that being said, I don't have a specific example where the latter limit exists at a point but the function is not twice differentiable there.
A: The question appears to be is there a difference approximation involving a single increment $h$ such that the approximation converges to the value of the second derivative as $h \to 0$ if and only if the second derivative exists.
The definition of the second derivative at a point $x$ is
$$f''(x) = \lim_{h \to 0}\frac{f'(x+h)- f'(x)}{h},$$
assuming that the first derivative is defined in a neighborhood of $x$.
As suggested in another answer this can be written as an iterated limit
$$f''(x) = \lim_{h \to 0}\lim_{k \to 0}\frac{f(x+h+k)- f(x+h) +f(x+k) + f(x)}{hk}.$$
Nothing has been proved yet as this is just a restatement of the definition.
The appropriate question now --  raised by the OP in comments -- is under what conditions does the double limit as $h,k \to 0$ converge to the same value:
$$f''(x) = \lim_{h,k \to 0}\frac{f(x+h+k)- f(x+h) -f(x+k) + f(x)}{hk}.$$
If so, then the diagonal limit with $h = k$ must converge and we have
$$f''(x) = \lim_{h \to 0}\frac{f(x+2h)- 2f(x+h) + f(x)}{h^2}.$$
Note that the iterated limits can be switched, but this is merely a consequence of symmetry and does not guarantee that the double limit exists.  There are, of course, many well known examples where the iterated limits converge to the same value but the double limit fails to exist.
If, however, the inner limit is uniformly convergent then the double limit exists. This is guaranteed when the second derivative is bounded in some neighborhood of $x$.  Then we have by Taylor's theorem
$$f(x+h+k) = f(x+h) + f'(x+h)k + f''(\xi)k^2/2, $$
and
$$\left|\frac{f(x+h+k) - f(x+h)}{k} - f'(x+h)\right| =  |f''(\xi)k/2| \leqslant Mk/2. $$
Then the LHS is uniformly convergent to $0$ as $k \to 0$ for all $x+h$ 
in the neighborhood where the second derivative is bounded.
A bounded second derivative in a neighborhood is, therefore, a sufficient condition for the forward difference approximation (and the central difference approximation) to converge to the correct value in conjunction with existence of the second derivative.
The signum function is a somewhat of a distraction here.  Since the second derivative fails to exist at $x = 0$ then the convergence of the central difference to $0$ while surprising is not relevant.  It is purely an artifact of the cancelation of terms for this particular function. In this case, the forward approximation will diverge to $\infty$ as one might expect.  
