Deriving things like chain rule I wanted to see how much calculus I could derive from the ground up using basic definitions.
I was able to show that for example
\begin{align}
\frac{d}{dx}cx^n &= \lim_{h \to 0} \frac{c \cdot(x+h)^n - cx^n}{h}
\\&= \lim_{h \to 0} \frac{c}{h} \cdot((x+h)^n - x^n)
\\&= \lim_{h \to 0} \frac{c}{h} \cdot\left(-x^n + \sum_{k=0}^{n}\binom{n}{k}x^kh^{n-k}\right)
\\&= \lim_{h \to 0} \frac{c}{h} \cdot\left(\sum_{k=0}^{n-1}\binom{n}{k}x^kh^{n-k}\right)
\\&= \lim_{h \to 0} \frac{c}{h} \cdot\left(nh\sum_{k=0}^{n-1}\binom{n-1}{k}x^kh^{n-1-k}\right)
\\&= \lim_{h \to 0} \frac{c}{h} \cdot\left(nh(x+h)^{n-1}\right)
\\&= \lim_{h \to 0} cn(x+h)^{n-1}
\\&= cnx^{n-1}
\end{align}
Which surprised me that it actually worked!
However when I tried the same thing for chain rule I was hopelessly stuck:
\begin{align}
\frac{d}{dx}f(g(x))&= \lim_{h \to 0} \frac{f(g(x+h)) - f(g(x))}{h}
\end{align}
And sadly that is as far as I could go! I have no idea how you're supposed to simplify that any further.
Or should it be 
\begin{align}
\frac{d}{dx}f(g(x))&= \lim_{h \to 0} \frac{f(g(x)+h) - f(g(x))}{h}
\end{align}
instead?
 A: Chain rule is a tricky thing to prove; the proof follows the intuition of "cancellation" in $\frac{dy}{dx}= \frac{dy}{du}\frac{du}{dx}$. In particular, rewrite your limit as
$$
\lim_{h\to 0} \frac{f(g(x+h))-f(g(x))}{g(x+h)-g(x)}
\frac{g(x+h) - g(x)}{h}
$$
Getting that first term into the desired form from here requires a careful treatment of the definition of a limit and continuity. Essentially, we need to argue that if we set $u(h)=g(x+h)$, then the limit of the first fraction can be rewritten as
$$
\lim_{u\to g(x)} \frac{f(u) - f(g(x))}{u-g(x)}
$$
Which is to say that the limit will be $f'(g(x))$.
A: This one takes some thought, but is essentially just applying the definition of the derivative.
Put $h(x) = g(f(x))$. 
By the Mean Value Theorem, we have
$f(t) - f(x) = [t-x][f'(x) + u(t)],$ where $u(t) \to 0$ as $t \to x$,
and, setting $y = f(x)$
$g(f(t)) - g(f(x)) = [f(t) - f(x)][g'(f(x)) + v(f(t)]$, where $v(f(t)) \to 0$ as $f(t) \to f(x)$.
Then we have $h(t) - h(x) = g(f(t))- g(f(x)) = [f(t) - f(x)][g'(f(x)) + v(f(t)] = [t-x][f'(x) + u(t)][g'(f(x)) + v(f(t)]$.
Then, letting $t \to x$, we see that $f(t) \to f(x)$, hence $v(f(t)) \to 0$ and $u(t) \to 0$. 
Dividing through by $t-x$ and letting $t \to x$, we arrive at $
lim_{t \to x}\frac {h(t) - h(x)}{t-x} = f'(x)g'(f(x))$, as desired.
A: We have
$$
\begin{align}
\frac{f(g(x+h)) - f(g(x))}{h}&=\frac{f(g(x+h)) - f(g(x))}{g(x+h)-g(x)}\cdot\frac{g(x+h)-g(x)}{h}
\end{align}
$$
where clearly the last fraction converges to $g'(x)$. Then notice that for $g(x+h)\neq g(x)$ the first fraction is the slope of a secant on the graph of $f$, but it is undefined for $g(x+h)=g(x)$ due to division by zero. By defining
$$
\frac{\Delta f}{\Delta g}(h)=
\begin{cases}
\frac{f(g(x+h)) - f(g(x))}{g(x+h)-g(x)}&\text{if }g(x+h)\neq g(x)\\
\quad\\
f'(g(x))&\text{if }g(x+h)=g(x)
\end{cases}
$$
it is straightforward to show that $\frac{\Delta f}{\Delta g}(h)$ is a continuous function of $h$ (because $f$ is differentiable at $g(x)$), and that
$$
\begin{align}
\frac{f(g(x+h)) - f(g(x))}{h}&=\frac{\Delta f}{\Delta g}(h)\cdot\frac{g(x+h)-g(x)}{h}
\end{align}
$$
clearly holds when $g(x+h)\neq g(x)$, but also when $g(x+h)=g(x)$ since then both sides become zero.
