1
$\begingroup$

I'm working with a proof of the multivariable chain rule $\frac{d}{dt}g(t)=\frac{df}{dx_1}\frac{dx_1}{dt}+\frac{df}{dx_2}\frac{dx_2}{dt}$ for $g(t)=f(x_1(t),x_2(t))$, but I have a hard time understanding two important steps of this proof.

The proof includes the function $\Delta_i(h)=x_i(t+h)-x_i(t)$ for i=1,2, $\bar{\Delta}=(\Delta_1(h),\Delta_2(h))$ $\Rightarrow$ $lim_{h\rightarrow0}\frac{\Delta_i}{h}=x^{'}_i$. It says that

$\frac{g(t+h)-g(t)}{h}=\frac{f(\bar{x}(t+h))-f(\bar{x}(t))}{h}=\frac{f(\bar{x}(t)-\bar{\Delta(h))}-f(\bar{x}(t))}{h}$

which I understand, but the next step is the to state that $f$ is differentiable and then let the previous equation be equal to

$=f^{'}_1(\bar{x}(t))\cdot\Delta_1(h)+f^{'}_2(\bar{x}(t))\cdot\Delta_2(h)+o(\vert\vert\bar{\Delta}\vert\vert)$

and this step I do not understand. I think there might be missing some limit-notation? But even with the limit notation I'm still not sure as to how it becomes a partial derivative multiplied with $\Delta_i$.

Afterwards they let $h\rightarrow0$ to get

$=f^{'}_1(\bar{x}(t))\cdot x_1^{'}(t)+f^{'}_2(\bar{x}(t))\cdot x_2^{'}(t)$

Again I am very confused as to possibly missing limit notations.

Does anyone know this version of the proof of the chain rule (besides these two steps, I find it the easiest version to understand), or understand these steps?

Here are pictures of the notes:

Theorem: Multivariable chain rule

Proof of theorem

$\endgroup$
  • $\begingroup$ See this page regarding the "little-oh notation" : en.wikipedia.org/wiki/Big_O_notation . Although it says Big-Oh, there is a section on little-oh on this page as well. $\endgroup$ – David Reed Dec 10 '17 at 9:31
  • $\begingroup$ Actually, I see the logic behind the little-oh - it's more the products of the partial derivaties and lambda-functions that I can not see how comes up :) $\endgroup$ – NKH Dec 10 '17 at 9:39
  • $\begingroup$ Well first,this is not really a proof but an informal argument. You won't see a real proof of either single or multivariate chain rules until you take real analysis. Could you upload a picture of this for me so I can see the entire thing? Also when you say lambda do you mean delta? $\endgroup$ – David Reed Dec 10 '17 at 9:56
  • $\begingroup$ Of course, sorry, I meant delta. I have now edited the question and uploaded a picture of the notes I have. $\endgroup$ – NKH Dec 10 '17 at 10:15
  • $\begingroup$ Consider that you are dealing wih: $$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$ $$g: \mathbb{R^n} \to \mathbb{R}$$ $$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$ To prove you only have to use the definition of differtial showing that the composition is also a differential. $\endgroup$ – user Dec 10 '17 at 10:43
1
$\begingroup$

Multivariable chain rule descends from the theorem of composite function for function of several variables which states in general that if:

f and g are differentiable in $x_0$ and $y_0=f(x_0)$, that is:

$$f(x_0+h)=f(x_0)+J_f(x_0)\cdot h+o(|h|)$$

$$g(y_0+k)=g(y_0)+J_g(y_0)\cdot k+o(|k|)$$

The composite function $g \circ f$ is also differentiable in $x_0$ and:

$$g(f(x_0+h))=g(f(x_0))+J_g(y_0)\cdot J_f(x_0)\cdot h+o(|h|)$$

NOTE

For the proof it is convenient to write:

$o(|h|)=|h|\cdot \omega_f(h)$ with $\omega_f(h) \to 0$

$o(|k|)=|k|\cdot \omega_g(k)$ with $\omega_g(k) \to 0$.

In the special case of:

$$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$

$$g: \mathbb{R^n} \to \mathbb{R}$$

$$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$

we have

$$J_f(t)= \begin{bmatrix} \frac{dx_1}{d t} \\ .\\\frac{dx_n}{d t} \end{bmatrix} = \begin{bmatrix} x_1' \\ .\\ x_n' \end{bmatrix}$$

$$J_g(x)= \nabla g = \left( \frac{\partial g}{\partial x_1},...,\frac{\partial g}{\partial x_n} \right)$$

And finally:

$$J_g(x)\cdot J_f(t)=\frac{\partial g}{\partial x_1}\frac{dx_1}{dt}+...+\frac{\partial g}{\partial x_n}\frac{dx_n}{dt}$$

That is the chain rule for this particular case.

Take also a look here: Derivation of the multivariate chain rule

The general theorem allow to find similar rules for any case by the Jacobian matrices $J_f$ and $J_g$.

$\endgroup$
  • $\begingroup$ Thank you, that helps a lot actually. But I'm still unsure about the derivative of this function. We now know $g(f(x_0+h))=g(f(x_0))+J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)$ but to take the derivative we need to write $lim_{h\rightarrow 0}\frac{g(f(x_0+h)-g(f(x_0))}{h}=lim_{h\rightarrow 0} \frac{J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)}{h}$ I understand that little-oh can be neglected but what happens to the Jacobian matrices? $\endgroup$ – NKH Dec 10 '17 at 10:48
  • $\begingroup$ In this case to understand deeply the rule it is necessary to generalize the context. The key point is that tha differential is a linear map, thus the composition is also a linear map i.e. a product of matrices /vectors (i.e.Jacobian, Gradient). $\endgroup$ – user Dec 10 '17 at 10:52
  • $\begingroup$ Sorry there was a typo, I missed the h, now it's fixed! $\endgroup$ – user Dec 10 '17 at 10:54
  • $\begingroup$ In the proof the onlydifficult part is to show that o(|h|)/h->0. $\endgroup$ – user Dec 10 '17 at 10:56
  • $\begingroup$ I've added some additional note in the answer. $\endgroup$ – user Dec 10 '17 at 11:06
2
$\begingroup$

These days I've been looking for a rigurous proof of the multivariable chain rule and I've finally found one that I think is very easy to understand. I will leave it here (if nobody minds) for anybody searching for this that is not familiar with little-o notation, Jacobians and stuff like this. To understand this proof, all you need to know is the mean value theorem.

Let's say we have a function $f(x,y)$ and $x = x(t), y = y(t)$. Let's also take $z(t) = f(x(t), y(t))$ By definition, the derivative of z $z'(t)$ is

$$ z'(t) = \lim_{\Delta t \to 0}{\frac {f(x(t+\Delta t),y(t+\Delta t)) - f(x,y)}{\Delta t}}$$.

$$ Let \ \Delta x = x(t+\Delta t)-x(t),$$ $$\Delta y = y(t+\Delta t)-y(t)$$

Now I'll take the numerator of the fraction in the limit, and make a small change.

$$ f(x(t+\Delta t), y(t+\Delta t)) - f(x,y) = f(x+\Delta x, y+\Delta y) - f(x,y)$$ $$ = \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right] + \left[f(x+\Delta x, y) - f(x, y)\right]$$

I have just added and substracted $f(x+\Delta x, y)$. For some reason, I will invert the terms.

$$ = \left[f(x+\Delta x, y) - f(x, y)\right] + \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right]$$.

Now, let's define 2 functions and I will name them g and h. First,

$$ Let \ g(x) = f(x, y) \implies g'(x) = \frac {\partial f} {\partial x} $$.

Please note that y is constant here since g is a function of a single variable. Now, by the mean value theorem we have

$$ \exists c_1 \in (x, x+\Delta x) \ so \ that$$ $$\frac {g(x+\Delta x) - g(x)} {\Delta x} = g'(c_1) $$ $$ \Longleftrightarrow $$ $$ f(x+\Delta x, y) - f(x, y) = f_x(c_1, y)\Delta x$$

Similarly, using the function $$ h(y) = f(x + \Delta x, y) \implies h'(y) = \frac {\partial} {\partial y}f(x+\Delta x, y)$$

We will have by the same logic that

$$ f(x+\Delta x, y + \Delta y) - f(x+\Delta x, y) = f_y(x + \Delta x, c_2)\Delta y, c_2 \in (y, y+\Delta y) $$

Notice that $c_1$ and $c_2$ are bounded with respect to $\Delta x$ and $\Delta y$ So as $\Delta x \to 0, c_1 \to x$ and as $\Delta y \to 0, c_2 \to y$. By our definition of $\Delta x$ and $\Delta y$, as $\Delta t \to 0$, both $\Delta x$ and $\Delta y$ $\to 0$. So, as $\Delta t \to 0$, $c_1 \to x$ and $c_2 \to y$.

The last step of the proof is to sum this all up, divide by $\Delta t$ and take the limit as $\Delta t \to 0$

$$ f(x(t+\Delta t), y(t+\Delta t)) - f(x, y) = f_x(c_1, y)\Delta x + f_y(x+\Delta x, c_2)\Delta y $$ $$ \lim_{\Delta t \to 0} \frac {f(x(t+\Delta t), y(t+\Delta t))}{\Delta t} = \lim_{\Delta t \to 0} f_x(c_1, y)\frac {\Delta x}{\Delta t} + f_y(x+\Delta x, c_2)\frac {\Delta y}{\Delta t} = f_x(x, y)x'(t) + f_y(x, y)y'(t) \ QED $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.