5
$\begingroup$

I'm working with a proof of the multivariable chain rule $\displaystyle{\frac{d}{dt}g(t)=\frac{df}{dx_1}\frac{dx_1}{dt}+\frac{df}{dx_2}\frac{dx_2}{dt}}$ for $g(t)=f(x_1(t),x_2(t))$, but I have a hard time understanding two important steps of this proof.

The proof includes the function $\displaystyle{\Delta_i(h)=x_i(t+h)-x_i(t)}$ for $\displaystyle{i=1,2, \bar{\Delta}=(\Delta_1(h),\Delta_2(h)) \Rightarrow \lim_{h\rightarrow0}\frac{\Delta_i}{h}=x^{'}_i}$. It says that

$\frac{g(t+h)-g(t)}{h}=\frac{f(\bar{x}(t+h))-f(\bar{x}(t))}{h}=\frac{f(\bar{x}(t)-\bar{\Delta(h))}-f(\bar{x}(t))}{h}$

which I understand, but the next step is the to state that $f$ is differentiable and then let the previous equation be equal to

$=f^{'}_1(\bar{x}(t))\cdot\Delta_1(h)+f^{'}_2(\bar{x}(t))\cdot\Delta_2(h)+o(\vert\vert\bar{\Delta}\vert\vert)$

and this step I do not understand. I think there might be missing some limit-notation? But even with the limit notation I'm still not sure as to how it becomes a partial derivative multiplied with $\Delta_i$.

Afterwards they let $h\rightarrow0$ to get

$=f^{'}_1(\bar{x}(t))\cdot x_1^{'}(t)+f^{'}_2(\bar{x}(t))\cdot x_2^{'}(t)$

Again I am very confused as to possibly missing limit notations.

Does anyone know this version of the proof of the chain rule (besides these two steps, I find it the easiest version to understand), or understand these steps?

Here are pictures of the notes:

Theorem: Multivariable chain rule

Proof of theorem

$\endgroup$
8
  • $\begingroup$ See this page regarding the "little-oh notation" : en.wikipedia.org/wiki/Big_O_notation . Although it says Big-Oh, there is a section on little-oh on this page as well. $\endgroup$
    – David Reed
    Dec 10, 2017 at 9:31
  • $\begingroup$ Actually, I see the logic behind the little-oh - it's more the products of the partial derivaties and lambda-functions that I can not see how comes up :) $\endgroup$
    – NKH
    Dec 10, 2017 at 9:39
  • $\begingroup$ Well first,this is not really a proof but an informal argument. You won't see a real proof of either single or multivariate chain rules until you take real analysis. Could you upload a picture of this for me so I can see the entire thing? Also when you say lambda do you mean delta? $\endgroup$
    – David Reed
    Dec 10, 2017 at 9:56
  • $\begingroup$ Of course, sorry, I meant delta. I have now edited the question and uploaded a picture of the notes I have. $\endgroup$
    – NKH
    Dec 10, 2017 at 10:15
  • $\begingroup$ Consider that you are dealing wih: $$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$ $$g: \mathbb{R^n} \to \mathbb{R}$$ $$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$ To prove you only have to use the definition of differtial showing that the composition is also a differential. $\endgroup$
    – user
    Dec 10, 2017 at 10:43

2 Answers 2

10
$\begingroup$

These days I've been looking for a rigurous proof of the multivariable chain rule and I've finally found one that I think is very easy to understand. I will leave it here (if nobody minds) for anybody searching for this that is not familiar with little-o notation, Jacobians and stuff like this. To understand this proof, all you need to know is the mean value theorem.

Let's say we have a function $f(x,y)$ and $x = x(t), y = y(t)$. Let's also take $z(t) = f(x(t), y(t))$ By definition, the derivative of z $z'(t)$ is

$$ z'(t) = \lim_{\Delta t \to 0}{\frac {f(x(t+\Delta t),y(t+\Delta t)) - f(x,y)}{\Delta t}}$$.

$$ Let \ \Delta x = x(t+\Delta t)-x(t),$$ $$\Delta y = y(t+\Delta t)-y(t)$$

Now I'll take the numerator of the fraction in the limit, and make a small change.

$$ f(x(t+\Delta t), y(t+\Delta t)) - f(x,y) = f(x+\Delta x, y+\Delta y) - f(x,y)$$ $$ = \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right] + \left[f(x+\Delta x, y) - f(x, y)\right]$$

I have just added and substracted $f(x+\Delta x, y)$. For some reason, I will invert the terms.

$$ = \left[f(x+\Delta x, y) - f(x, y)\right] + \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right]$$.

Now, let's define 2 functions and I will name them g and h. First,

$$ Let \ g(x) = f(x, y) \implies g'(x) = \frac {\partial f} {\partial x} $$.

Please note that y is constant here since g is a function of a single variable. Now, by the mean value theorem we have

$$ \exists c_1 \in (x, x+\Delta x) \ so \ that$$ $$\frac {g(x+\Delta x) - g(x)} {\Delta x} = g'(c_1) $$ $$ \Longleftrightarrow $$ $$ f(x+\Delta x, y) - f(x, y) = f_x(c_1, y)\Delta x$$

Similarly, using the function $$ h(y) = f(x + \Delta x, y) \implies h'(y) = \frac {\partial} {\partial y}f(x+\Delta x, y)$$

We will have by the same logic that

$$ f(x+\Delta x, y + \Delta y) - f(x+\Delta x, y) = f_y(x + \Delta x, c_2)\Delta y, c_2 \in (y, y+\Delta y) $$

Notice that $c_1$ and $c_2$ are bounded with respect to $\Delta x$ and $\Delta y$ So as $\Delta x \to 0, c_1 \to x$ and as $\Delta y \to 0, c_2 \to y$. By our definition of $\Delta x$ and $\Delta y$, as $\Delta t \to 0$, both $\Delta x$ and $\Delta y$ $\to 0$. So, as $\Delta t \to 0$, $c_1 \to x$ and $c_2 \to y$.

The last step of the proof is to sum this all up, divide by $\Delta t$ and take the limit as $\Delta t \to 0$

$$ f(x(t+\Delta t), y(t+\Delta t)) - f(x, y) = f_x(c_1, y)\Delta x + f_y(x+\Delta x, c_2)\Delta y $$ $$ \lim_{\Delta t \to 0} \frac {f(x(t+\Delta t), y(t+\Delta t))}{\Delta t} = \lim_{\Delta t \to 0} f_x(c_1, y)\frac {\Delta x}{\Delta t} + f_y(x+\Delta x, c_2)\frac {\Delta y}{\Delta t} = f_x(x, y)x'(t) + f_y(x, y)y'(t) \ QED $$

Edit: After a long time I've realised that this proof assumes that $f$ has partial derivatives defined on intervals around the point $(x, y)$ and they are continuous at the point. This is a sufficient condition for the function to be ($\mathbb{R}^2$-)differentiable at $(x, y)$, but it's not equivalent. Yet, the multivariable chain rule works for the function being just differentiable at that point. So for a general proof, one should first understand little-o notation as in the other answers.

$\endgroup$
4
  • $\begingroup$ Can you use this to prove for more than 2 variables? $\endgroup$
    – GLaDOS
    Apr 15, 2020 at 16:01
  • $\begingroup$ Yes, this generalises fairly easily to the multivariable chain rule: for a function $f({\bf x})$ where ${\bf x}$ is an n-dimensional vector, $\frac{d}{dt} f({\bf x}(t)) = \nabla f({\bf x}(t)) \cdot {\bf x}'(t)$ $\endgroup$ Apr 15, 2020 at 19:38
  • $\begingroup$ Very straightforward, yet rigorous proof. One small modification I would do is defining $c_i$ as $c_i = \theta_ix + (1-\theta_i)\Delta x$ for some $\theta_i\in(0,1)$. This way you make sure that the negative case is automatically covered. Also, curious whether this proof also covers the cases where e.g. $\Delta x$ rapidly takes value of $0$ close to the limit point, such as when $x(t) = x^2\sin(1/x)$. $\endgroup$
    – V.S.e.H.
    Aug 27, 2020 at 20:28
  • $\begingroup$ @Viktorio El Hakim Both issues can be covered (I think) by letting $c_1 = \left\{\begin{align*}&x + (1-\theta_1)\Delta x & \Delta x \ne 0 \\ &x & \Delta x = 0 \end{align*}\right.$ This way, $g(x+\Delta x) - g(x) = g'(c_1)\Delta x$ is still true, MVT covering the first branch and $0 = 0$ the second one, and $\lim_{t\to 0} c_1(t) = x$. I'm not sure if there are any flaws to this, thanks for pointing those out. $\endgroup$ Aug 28, 2020 at 14:42
4
$\begingroup$

Multivariable chain rule descends from the theorem of composite function for function of several variables which states in general that if:

f and g are differentiable in $x_0$ and $y_0=f(x_0)$, that is:

$$f(x_0+h)=f(x_0)+J_f(x_0)\cdot h+o(|h|)$$

$$g(y_0+k)=g(y_0)+J_g(y_0)\cdot k+o(|k|)$$

The composite function $g \circ f$ is also differentiable in $x_0$ and:

$$g(f(x_0+h))=g(f(x_0))+J_g(y_0)\cdot J_f(x_0)\cdot h+o(|h|)$$

NOTE

For the proof it is convenient to write:

$o(|h|)=|h|\cdot \omega_f(h)$ with $\omega_f(h) \to 0$

$o(|k|)=|k|\cdot \omega_g(k)$ with $\omega_g(k) \to 0$.

In the special case of:

$$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$

$$g: \mathbb{R^n} \to \mathbb{R}$$

$$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$

we have

$$J_f(t)= \begin{bmatrix} \frac{dx_1}{d t} \\ .\\\frac{dx_n}{d t} \end{bmatrix} = \begin{bmatrix} x_1' \\ .\\ x_n' \end{bmatrix}$$

$$J_g(x)= \nabla g = \left( \frac{\partial g}{\partial x_1},...,\frac{\partial g}{\partial x_n} \right)$$

And finally:

$$J_g(x)\cdot J_f(t)=\frac{\partial g}{\partial x_1}\frac{dx_1}{dt}+...+\frac{\partial g}{\partial x_n}\frac{dx_n}{dt}$$

That is the chain rule for this particular case.

Take also a look here: Derivation of the multivariate chain rule

The general theorem allow to find similar rules for any case by the Jacobian matrices $J_f$ and $J_g$.

$\endgroup$
7
  • $\begingroup$ Thank you, that helps a lot actually. But I'm still unsure about the derivative of this function. We now know $g(f(x_0+h))=g(f(x_0))+J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)$ but to take the derivative we need to write $lim_{h\rightarrow 0}\frac{g(f(x_0+h)-g(f(x_0))}{h}=lim_{h\rightarrow 0} \frac{J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)}{h}$ I understand that little-oh can be neglected but what happens to the Jacobian matrices? $\endgroup$
    – NKH
    Dec 10, 2017 at 10:48
  • $\begingroup$ In this case to understand deeply the rule it is necessary to generalize the context. The key point is that tha differential is a linear map, thus the composition is also a linear map i.e. a product of matrices /vectors (i.e.Jacobian, Gradient). $\endgroup$
    – user
    Dec 10, 2017 at 10:52
  • $\begingroup$ Sorry there was a typo, I missed the h, now it's fixed! $\endgroup$
    – user
    Dec 10, 2017 at 10:54
  • $\begingroup$ In the proof the onlydifficult part is to show that o(|h|)/h->0. $\endgroup$
    – user
    Dec 10, 2017 at 10:56
  • $\begingroup$ I've added some additional note in the answer. $\endgroup$
    – user
    Dec 10, 2017 at 11:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .