I've been working on understanding limits thoroughly, so I'm rewriting how I understand the chain rule. Please help me fill in my gaps in understanding.

$f$ is some function. Then

$f'(x) = \lim\limits_{h \to 0}\frac{f(x+h)-f(x)}{h}$

Now I might want to evaluate something like

$$\left(f(g(x))\right)' = \lim\limits_{h \to 0}\frac{f(g(x+h))-f(g(x))}{h}$$

Evaluating this is tricky, so we need a way to do it.

$\left(f(g(x))\right)' = (\lim\limits_{h \to 0}\frac{f(g(x+h))-f(g(x))}{g(x+h)-g(x)}\cdot\frac{g(x+h)-g(x)}{h}$)

$\left(f(g(x))\right)' = (\lim\limits_{h \to 0}\frac{f(g(x+h))-f(g(x))}{g(x+h)-g(x)})\cdot\lim\limits_{h \to 0}(\frac{g(x+h)-g(x)}{h})$

if $k=g(x+h)-g(x)$, then

$g(x+h)=k+g(x)$, so

$\left(f(g(x))\right)' = (\lim\limits_{h \to 0}\frac{f(g(x)+k)-f(g(x))}{k})\cdot(\lim\limits_{h \to 0}\frac{g(x+h)-g(x)}{h})$

and, assuming I can go ahead and just change the limit variable on the left term, then

$=(\lim\limits_{k \to 0}\frac{f(g(x)+k)-f(g(x))}{k}) \cdot (\lim\limits_{h \to 0}\frac{g(x+h)-g(x)}{h})$

$=f'(g(x))\cdot g'(x)$

Which is easier to figure out, and is also the chain rule.

Is that correct?

  • $\begingroup$ You switch from treating $k$ as a function to treating $k$ as a constant, which is unjustified. $\endgroup$ – Alex Becker Jan 6 '12 at 0:17
  • $\begingroup$ Where is $k$ constant? $\endgroup$ – Korgan Rivera Jan 6 '12 at 0:21
  • $\begingroup$ When you take the limit as $k$ goes to zero. In principle I suppose you could do that with a function, but it requires more bookkeeping. I'd say you simply shouldn't use $k$ at all, it just makes it easy to forget that $k = k(x,h) = g(x+h)-g(x)$. $\endgroup$ – Alex Becker Jan 6 '12 at 0:29
  • $\begingroup$ The statement that $b\lim_{h\to0} (a/h) = \lim_{h\to0} (ba/h)$ assumes $b$ is constant. What you really need is $\lim_{h\to0} (ab) = (\lim_{h\to0}a)(\lim_{h\to0}b)$, which can be proved by $\varepsilon$-$\delta$ methods. $\endgroup$ – Michael Hardy Jan 6 '12 at 0:30
  • 1
    $\begingroup$ There is an error in the equality after "(which I don't know how to prove yet)": you cannot switch the $h$, which is the variable over which you are taking the limit, to outside the limit. $\endgroup$ – Arturo Magidin Jan 6 '12 at 1:01

The final idea is generally correct, as Michael Hardy points out, modulo some refinements to deal with tricky situations.

However, I wanted to point out that some of the arguments leading up to that final part are incorrect:

  • You assume that $$b\lim_{h\to 0}\frac{a}{h} = \lim_{h\to 0}\frac{ab}{h}.$$ This will work if $b$ does not depend on $h$ (in the sense that either both sides exist and are equal, or both sides do not exist), but cannot hold if $b$ does depend on $h$. To see that it cannot hold when $b$ depends on $h$, note that the right hand side will not depend on $h$ (because it's the value of a limit over $h$), but the left hand side does depend on $h$ (since $b$ depends on $h$ and is outside the limit, so the left hand side will be a function of $h$, namely $b$, times a constant, namely the value of the limit).

  • For the same reason, when you write: $$ \left(\lim\limits_{h \to 0}\frac{f(g(x)+k)-f(g(x))}{h}\right) \cdot \frac{k}{k} =\left(\lim\limits_{h \to 0}\frac{f(g(x)+k)-f(g(x))}{k}\right) \cdot \frac{k}{h}$$ you run into trouble, because you cannot just pull the $h$ out of the limit that is a limit over $h$, and because $k$ depends on $h$, so you cannot move it in and out of a limit over $h$ as you would a constant.

What you need in both cases is the "product rule for limits": if $$\lim_{h\to 0}\;k\quad\text{and}\quad \lim_{h\to 0}\;m\quad\text{both exist, then }\lim_{h\to 0}\;km = \left(\lim_{h\to 0}\;k\right)\left(\lim_{h\to 0}\;m\right),$$ which is what you use in the final paragraphs.

We want to find $$\lim_{h\to 0}\frac{f(g(x+h)) - f(g(x))}{h}.$$

You propose defining $k(h)$ (and it's important to keep the dependence on $h$ explicit, to prevent the errors you fell into) by $$k(h) = g(x+h)-g(x)$$ so that we can rewrite $$\frac{f(g(x+h)) - f(g(x))}{h} = \frac{f(g(x)+k(h)) - f(g(x))}{h}.$$ Nothing wrong here. Intuitively, what we want to then do is rewrite again into $$\begin{align*} \frac{f(g(x)+k(h)) - f(g(x))}{h} &= \frac{f(g(x)+k(h)) -f(g(x))}{k(h)}\cdot\frac{k(h)}{h}\\ &= \frac{f(g(x)+k(h)) - f(g(x))}{k(h)}\cdot \frac{g(x+h)-g(x)}{h}, \end{align*}$$ and then use the fact that a limit of a product is the product of the limits (if they both exist), and a change of variable in the first limit from $h$ to $k$, to deduce the Chain Rule.

The problem with this is that the rewriting is only valid if $k(h)\neq 0$ when $h\neq 0$; otherwise, the two functions are not equal, since they don't have the same domain. That is, we know that $k(h)=g(x+h)-g(x)$ is $0$ when $h=0$, but it's possible for $k(h)$ to equal zero for other values of $h$; and at those values, $\frac{f(g(x)+k(h))-f(g(x))}{h}$ is defined, but $\frac{f(g(x)+k(h))-f(g(x))}{k(h)}\cdot\frac{g(x+h)-g(x)}{h}$ is not, and we don't have equality.

If you think about it, though, at the places where $k(h)=0$, we have that $$\frac{f(g(x)+k(h))-f(g(x))}{h}=0.$$ So the simple way to make this work is to use a different function instead of $$\frac{f(g(x)+k(h)-f(g(x))}{k(h)}$$ on the right hand side, one that is equal to $f'(g(x))$ when $k(h)=0$ (the value we want to get), and is equal to the old value when $k(h)\neq 0$.

So we define a function $G(h)$ as follows: $$G(h) = \left\{\begin{array}{ll} \frac{f(g(x)+k(h))- f(g(x))}{k(h)} &\text{if }k(h)\neq 0\\ f'(g(x))&\text{if }k(h)=0. \end{array}\right.$$ With this function, we do have that $$\frac{f(g(x+h)) - f(g(x))}{h} = G(h)\cdot\frac{g(x+h)-g(x)}{h}$$ for all $h$. Since they take the same values at all values of $h$ (except $h=0$, where they are both undefined), the two have the same limit as $h\to 0$: $$\lim_{h\to 0}\left(\frac{f(g(x+h))-f(g(x))}{h} \right)= \lim_{h\to 0}\left(G(h)\cdot\frac{g(x+h)-g(x)}{h}\right).$$ Now, $$\lim_{h\to0}\frac{g(x+h)-g(x)}{h} = g'(x),$$ so if $\lim\limits_{h\to 0}G(h)$ exists, then we'll be set.

Now, $k(h) = g(x+h)-g(x)$; as $h\to 0$, we have $k(h)\to 0$ because $g$ is continuous at $x$ (by virtue of being differentiable at $x$).

If $k(h)=0$ for all values of $h$ near $0$, then $$\lim\limits_{h\to 0}G(h) = \lim_{h\to 0}f'(g(x)) = f'(g(x)).$$

If $k(h)\neq 0$ for all $h$ near $0$ (except at $h=0$), then we can do a change of variable: since $\lim\limits_{h\to 0}k(h) = 0$, we have: $$\begin{align*} \lim_{h\to 0}G(h) &= \lim_{h\to0}\frac{f(g(x)+k(h)) - f(g(x))}{k(h)}\\ &= \lim_{k(h)\to 0}\frac{f(g(x)+k(h)) - f(g(x))}{k(h)}\\ &= f'(g(x)). \end{align*}$$ The justification for this formally requires epsilon and deltas, but the idea is: we can make $k(h)$ arbitrarily close to $0$ by making $h$ arbitrarily close to $0$, so taking the limit as $h$ approaches $0$ amounts to the same as taking the limit as $k(h)$ approaches zero.

What if $k(h)$ is not constant zero, but takes the value $0$ at arbitrarily small values of $h$ (that is, for every $\delta\gt 0$ we can find $h$ in $(-\delta,\delta)$ where $k(h)=0$, and we can find $h'$ in $(-\delta,\delta)$ where $k(h)\neq 0$)? Then one needs to argue carefully that the limit $$\lim_{h\to 0}G(h)$$ is still equal to $f'(g(x))$; this is difficult to do informally, and there are no ready "limit rules" to help us. You would need to see the proof with epsilon and deltas. One can think of this as a rather "extreme and almost pathological" case, and the easy cases above follow the intuition you had fairly well, once you fix the problems with your manipulation of limits.

Taking this for granted, we have: $$\begin{align*} \lim_{h\to 0}\frac{f(g(x+h))-f(g(x))}{h} &= \lim_{h\to 0}\frac{f(g(x)+k(h)) - f(g(x))}{h}\\ &= \lim_{h\to 0}\left(G(h)\cdot\frac{g(x+h)-g(x)}{h}\right)\\ &= \left(\lim_{h\to 0}G(h)\right)\left(\lim_{h\to 0}\frac{g(x+h)-g(x)}{h}\right)\\ &= f'(g(x))g'(x). \end{align*}$$

  • $\begingroup$ I appreciate everyone's input but nothing anyone has said has filled the gaps for me. (Please no jokes about how the gap may be between my ears :) Can someone go ahead and write the chain rule in the same way I did, only correctly? Don't skip any steps, make everything clear and explicit as if explaining to a child. I understand how the product rule for limits is useful here, but I need a rule for when the limits are of different variables, as in $\left(\lim_{a\to 0}\;k\right)\left(\lim_{b\to 0}\;m\right)$ $\endgroup$ – Korgan Rivera Jan 6 '12 at 3:56
  • $\begingroup$ @Korgan: There is no such rule. $\endgroup$ – Arturo Magidin Jan 6 '12 at 4:03
  • $\begingroup$ Thank you. I think I figured it out, and then I read your generous reply. I will use your response to weed out any bugs that are in my new idea. Thanks! $\endgroup$ – Korgan Rivera Jan 6 '12 at 5:46
  • $\begingroup$ Is the derivative of $f(g(x))$ = $f'(g(x))$ or $(f(g(x)))'$? You edited it to be the latter, though I thought the former was fine. $\endgroup$ – Korgan Rivera Jan 6 '12 at 5:55
  • 1
    $\begingroup$ @Korgan: $f'(g(x))$ is the derivative of $f$, evaluated at $g(x)$. This is not the derivative of $f(g(x))$ unless $g'(x)=1$. The Chain Rule says: $$(f(g(x)))' = f'(g(x))g'(x).$$ Note that "$f'(g(x))$" on the right hand side is not the derivative you are looking for (the derivative of $f\circ g$). What is valid notation is $(f\circ g)'(x)$, or $(f(g(x)))'$. You want the derivative of the compositum, not the derivative of $f$ evaluated at $g(x)$. $\endgroup$ – Arturo Magidin Jan 6 '12 at 5:58

This is correct except that it fails to address the fact that $k$ may be $0$ when $h$ is not. If there is some neighborhood of $0$ such that when $h$ is in that neighborhood and $h\ne0$, then $k\ne0$, then everything above works fine. If every open neighborhood of $0$ contains some value of $h\ne0$ for which $k=0$, then your argument fails to deal with that situation. This is about the behavior of the function $g$, not about $f$.

The fraction $$ \frac{f(g(x)+k)-f(g(x))}{k} $$ is undefined when $k=0$, which may happen in some cases even when $h=0$. Now consider the piecewise defined function $$ k \mapsto \begin{cases} \frac{f(g(x)+k)-f(g(x))}{k} & \text{if }k\ne 0, \\ \\ f'(g(x)) & \text{if } k = 0.\end{cases} \tag{2} $$ I think working with (2) in place of (1) can fill in the gap. You still probably have to worry about the details of the logic, but that's the main idea.

  • $\begingroup$ "That expression is undefined for values of h≠0 for which g(x+h) is equal to g(x)." Do you need to correct that? $\endgroup$ – Korgan Rivera Jan 6 '12 at 0:41
  • $\begingroup$ @KorganRivera : Yes. I've fixed it. I needed to deal with $f$, not with $g$; that's the one taht's problematic. $\endgroup$ – Michael Hardy Jan 6 '12 at 2:54
  • $\begingroup$ ....i.e. the behavior of $g$ may cause problems affecting the expression that approaches the derivative of $f$. $\endgroup$ – Michael Hardy Jan 6 '12 at 2:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.