Proof of the delta method The classical, well known delta method states the following:
If $\sqrt{n}(X_{n}-\theta)\overset{law}{\longrightarrow}N(0,\sigma^{2})$. Then the following holds:
$\sqrt{n}(g(X_{n})-g(\theta))\overset{law}{\longrightarrow}N(0,\sigma^{2}(g'(\theta))^{2})$ for any function $g$ satisfying the property that $g'(\theta)$ exists and is non-zero valued. 
The key step, proving this result, is the following expression:
$g(X_{n})=g(\theta)+g'(\overline{\theta})(X_{n}-\theta)$ for some intermediate value $\overline{\theta}$ with $X_{n}<\overline{\theta}<\theta$. 
What exactly ensures the existence of such a $\overline{\theta}$? It should follow using Taylor's theorem, but I am not able to argue rigorously.
 A: All that is needed is the differentiability of $g$ at the single point $\theta$; the intermediate value theorem and $\overline\theta$ are not needed.  Recall that "$g$ is differentiable at $\theta$" means that the limit $\lim_{h\to0} (g(\theta+h)-g(\theta))/h$ exists; we give its value the name $g'(\theta).$  Now define the function $r$ by setting $r(h) = (g(\theta+h)-g(\theta))/h - g'(\theta)$ if $h\neq 0$ and $r(0)=0$.  What we know about $r$ is more-or-less only that $\lim_{h\to0} r(h)=0.$ 
But now we have a degree 1 "Taylor approximation": $$g(\theta+h) = g(\theta) + g'(\theta)h + hr(h)$$ just by unrolling the definition of $r$, where we know $r(h)\to0$ as $h\to0$.  Apply this to $h=X_n-\theta$, and multiply by $\sqrt n$ to get $$ \sqrt n (g(X_n)-g(\theta)) = g'(\theta) \sqrt n(X_n-\theta) + \sqrt n (X_n-\theta) r( X_n-\theta).$$
Now Slutsky's theorem kicks in: $\sqrt n (X_n-\theta)$ is tight (or $O_P(1)$, if you will) and it multiplies $r(X_n-\theta)$, which converges in probability to $0$, so the product converges to 0 in probability.
The "secret sauce" is that the proof of a degree 1 Taylor approximation amounts to no more than a recitation of the definition of differentiability at the  point of expansion.  Unlike higher degree Taylor approximations, where stronger hypotheses and arguments are needed.
