What is the source of $\frac{df(x)}{dx} = nx^{n-1}$

I've been reading the Michael Spivak's book on calculus lately and one thing that is bothering me is how he renders out proofs for various theorems without spending some time on the "golden facts" of how it works or the historical derivation of such a rule.

$$\frac{df(x)}{dx} = nx^{n-1}$$

I am currently searching all over the place to find a bit more how we came from the limit definition to this simple polynomial expression. Could someone shed some more light on this or recommend a book that delves deeper into the core of these mechanics? I'm not interested in just proofs of theorems, I'd like to explore how and why they work. I have no doubt that they work, so I'd more appreciate the why/how than some proof anyone can do.

This is exactly the reason I wanted to explore this book, unfortunately, it doesn't satisfy my particular curiosity regarding derivatives/differentiation. Everywhere I try, it seems like everyone is trying to avoid that particular connection I personally deem quite important. Like everyone just learned how to use them, with little care about how or why it works.

-
That's only true if $f(x) = x^n$. – Neal Apr 24 '12 at 22:46
@Neal True. But the rule can be expanded. I wish to understand the most basic notions, expanding it further is trivial (in comparison) then. – Jonathon Migraw Apr 24 '12 at 22:47
I'm not sure what you mean by 'expanded.' For really basic notions, you should pick up a real analysis book, or, if there is one, turn to the chapter with the "$\epsilon$s and $\delta$s." – Neal Apr 24 '12 at 22:47
Delta-epsilon limit proofs. – Joe Apr 24 '12 at 22:49
@Neal & Jay Electronica - Thank you. I hope it will illuminate more on the subject. What I meant by expanded is well, a bit free form, but using various other theorems for a higher order polynomial(which I can more-or-less comprehend intuitively): $f'(x) = na_nx^{n-1} + (n-1)a_{n-1}x^{n-2}$... Down to the constant. If that makes sense. – Jonathon Migraw Apr 24 '12 at 22:53

Let $f(x) = x^n$ and consider the limit definition of $f'(x)$. There are several versions of this definition, but I'll use $$f'(x) = \lim_{h \rightarrow 0} \frac{f(x+h)-f(x)}{h}.$$ Applying the definition to the function $f(x) = x^n$ gives $$f'(x) = \lim_{h \rightarrow 0} \frac{(x+h)^n - x^n}{h}.$$

The key is to simplify the fraction and then take the limit. When you expand $(x+h)^n$, you get the term $x^n$ just one time, but you get the term $x^{n-1} h$ in $n$ different ways. (Write the $n$ factors $(x+h)$ in a row and choose one of the $n$ factors to select $h$ from, thereby forcing you to select $x$ from the remaining $n-1$ factors in order to produce $x^{n-1}h$.) Every other term in the expansion contains at least $2$ powers of $h$ in it. Summarizing, we have proven that $$(x+h)^n = x^n + n x^{n-1} h + h^2 P(x,h).$$ for some polynomial $P(x,h)$. (The full expansion of $(x+h)^n$ is the celebrated Binomial Theorem.) Plugging this back into the original definition and performing some algebra gives $$f'(x) = \lim_{h \rightarrow 0} \left( n x^{n-1} + h P(x,h) \right) = n x^{n-1}.$$

As far as I understand, this is how the rule was first discovered, albeit written in modern notation for clarity.

-

Every rule for finding derivatives of real-valued functions comes from the definition of the derivative: If $f:\mathbb{R}\to\mathbb{R}$, then $$f'(x) = \lim_{\tau\to x}\frac{f(\tau)-f(x)}{\tau-x}$$ provided this limit exists.

Let's assume you've built up the necessary limit rules.

Now the rule that Spivak is glibly citing can be proven from this definition using algebra and the limit rules. For example, let $f(x) = x^n$ for some integer $n$. Then $\tau^n - x^n = (\tau - x)\sum_{k=0}^{n-1}\tau^{n-k-1}x^k$, so since there is a punctured neighborhood of $x$ in which the function $g(\tau) = \frac{f(\tau)-f(x)}{\tau - x}$ agrees with $\sum_{k=0}^{n-1}\tau^{n-k-1}x^k$, and being a little pedantic, we have that \begin{align*}\lim_{\tau\to x}g(\tau) &= \lim_{\tau\to x}\frac{f(\tau)-f(x)}{\tau - x} \\ &= \lim_{\tau\to x}\sum_{k=0}^{n-1}\tau^{n-k-1}x^k \\ &= \sum_{k=0}^{n-1} \lim_{\tau\to x}(\tau^{n-k-1}x^k) \\ &= \sum_{k=0}^{n-1} x^k(\lim_{\tau \to x}(\tau^{n-k-1}) \\ &= \sum_{k=0}^{n-1} x^k(\lim_{\tau\to x}\tau)^{n-k-1} \\ &= \sum_{k=0}^{n-1}x^kx^{n-k-1} \\ &= \sum_{k=0}^{n-1} x^{n-1} \\ &= nx^{n-1}.\end{align*}

So, for polynomials at least, all that's required is algebra and knowledge of how the limit rules work. To prove other facts, like $(e^x)' = e^x$, you need to have some grasp of the definition of the limit, $\lim_{\tau\to x}f(x) = L$ if and only if for every $\epsilon > 0$ there exists some $\delta>0$ so that $|\tau - x|<\delta$ implies that $|f(\tau) - L|<\epsilon$. You use this definition to build up the rules of limits, and to prove facts about transcendental functions which are not amenable to algebra.

-
I took the liberty of replacing \begin{eqnarray*} with \begin{align*} in order to have the equality signs aligned in a vertical column. If you want to use \begin{eqnarray*} use & = & instead of only & = (the alignment in eqnarray is right aligned & centered & left aligned). – t.b. Apr 24 '12 at 23:15
Thank you. I'll keep that in mind for the future. – Neal Apr 25 '12 at 14:07

Here is the proof with first principles:

$$y=x^n$$

$$\frac{dy}{dx}=\lim_{h \rightarrow 0}\frac{f(x+h)-f(x)}{h}=\lim_{h \rightarrow 0}\frac{(x+h)^{n}-x^n}{h} =$$

Using Bionmial Theorem:

$$\lim_{h \rightarrow 0}\frac{\sum_{k=0}^{n}(^n_k)x^{n-k}h^{k}-x^n}{h}=\lim_{h \rightarrow 0}\frac{x^n+nx^{n-1}h+(^n_2)nx^{n-2}h^2 + \cdots+nxh^{n-1}+h^n-x^n}{h}=$$ $$\lim_{h \rightarrow 0}\frac{nx^{n-1}h+(^n_2)nx^{n-2}h^2 + \cdots+nxh^{n-1}+h^n}{h} =$$ $$\lim_{h \rightarrow 0} nx^{n-1}+(^n_2)nx^{n-2}h + \cdots+nxh^{n-1}+h^{n-1} =$$ $$nx^{n-1}$$

Another, simpler definition (without first principles, so this may not be helpful):

$$y=x^n$$

$$\log y = n \log x$$

Taking the derivative of both sides:

$$\frac{y'}{y} = \frac{n}{x}\Rightarrow y' = y\frac{n}{x}= \frac{nx^n}{x}=nx^{n-1}$$

-

The binomial-theorem version of the proof is overkill, since the binomial theorem gives all of the coefficients in the expansion of $(x+\Delta x)^n$ where only two are needed. The version that says $$\lim_{w\to x} \frac{w^n-x^n}{w-x} = \lim_{w\to x} \frac{(w-x)(w^{n-1} + x^{n-2}x +w^{n-3}x^2 +\cdots + x^{n-1})}{w-x}$$ etc., is not overkill in that sense.

Here's another argument: The rule is easy to prove when $n=0$. So let $(d/dx)x^n=nx^{n-1}$ be the induction hypothesis and then write $$\frac{d}{dx} x^{n+1} = \frac{d}{dx} (x\cdot x^n)$$ and apply the product rule and then simplify.

-
Ha! Using the expansion of $w^n-x^n$ seems much nicer than using the binomial theorem. I won't forget this one for future uses! Also, as Spivak does, induction with this type of problems comes very handy. – Pedro Tamaroff Apr 25 '12 at 2:02