# Proof of multivariable chain rule

I'm working with a proof of the multivariable chain rule $\frac{d}{dt}g(t)=\frac{df}{dx_1}\frac{dx_1}{dt}+\frac{df}{dx_2}\frac{dx_2}{dt}$ for $g(t)=f(x_1(t),x_2(t))$, but I have a hard time understanding two important steps of this proof.

The proof includes the function $\Delta_i(h)=x_i(t+h)-x_i(t)$ for i=1,2, $\bar{\Delta}=(\Delta_1(h),\Delta_2(h))$ $\Rightarrow$ $lim_{h\rightarrow0}\frac{\Delta_i}{h}=x^{'}_i$. It says that

$\frac{g(t+h)-g(t)}{h}=\frac{f(\bar{x}(t+h))-f(\bar{x}(t))}{h}=\frac{f(\bar{x}(t)-\bar{\Delta(h))}-f(\bar{x}(t))}{h}$

which I understand, but the next step is the to state that $f$ is differentiable and then let the previous equation be equal to

$=f^{'}_1(\bar{x}(t))\cdot\Delta_1(h)+f^{'}_2(\bar{x}(t))\cdot\Delta_2(h)+o(\vert\vert\bar{\Delta}\vert\vert)$

and this step I do not understand. I think there might be missing some limit-notation? But even with the limit notation I'm still not sure as to how it becomes a partial derivative multiplied with $\Delta_i$.

Afterwards they let $h\rightarrow0$ to get

$=f^{'}_1(\bar{x}(t))\cdot x_1^{'}(t)+f^{'}_2(\bar{x}(t))\cdot x_2^{'}(t)$

Again I am very confused as to possibly missing limit notations.

Does anyone know this version of the proof of the chain rule (besides these two steps, I find it the easiest version to understand), or understand these steps?

Here are pictures of the notes:

Theorem: Multivariable chain rule

Proof of theorem

• See this page regarding the "little-oh notation" : en.wikipedia.org/wiki/Big_O_notation . Although it says Big-Oh, there is a section on little-oh on this page as well. – David Reed Dec 10 '17 at 9:31
• Actually, I see the logic behind the little-oh - it's more the products of the partial derivaties and lambda-functions that I can not see how comes up :) – NKH Dec 10 '17 at 9:39
• Well first,this is not really a proof but an informal argument. You won't see a real proof of either single or multivariate chain rules until you take real analysis. Could you upload a picture of this for me so I can see the entire thing? Also when you say lambda do you mean delta? – David Reed Dec 10 '17 at 9:56
• Of course, sorry, I meant delta. I have now edited the question and uploaded a picture of the notes I have. – NKH Dec 10 '17 at 10:15
• Consider that you are dealing wih: $$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$ $$g: \mathbb{R^n} \to \mathbb{R}$$ $$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$ To prove you only have to use the definition of differtial showing that the composition is also a differential. – user Dec 10 '17 at 10:43

Multivariable chain rule descends from the theorem of composite function for function of several variables which states in general that if:

f and g are differentiable in $x_0$ and $y_0=f(x_0)$, that is:

$$f(x_0+h)=f(x_0)+J_f(x_0)\cdot h+o(|h|)$$

$$g(y_0+k)=g(y_0)+J_g(y_0)\cdot k+o(|k|)$$

The composite function $g \circ f$ is also differentiable in $x_0$ and:

$$g(f(x_0+h))=g(f(x_0))+J_g(y_0)\cdot J_f(x_0)\cdot h+o(|h|)$$

NOTE

For the proof it is convenient to write:

$o(|h|)=|h|\cdot \omega_f(h)$ with $\omega_f(h) \to 0$

$o(|k|)=|k|\cdot \omega_g(k)$ with $\omega_g(k) \to 0$.

In the special case of:

$$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$

$$g: \mathbb{R^n} \to \mathbb{R}$$

$$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$

we have

$$J_f(t)= \begin{bmatrix} \frac{dx_1}{d t} \\ .\\\frac{dx_n}{d t} \end{bmatrix} = \begin{bmatrix} x_1' \\ .\\ x_n' \end{bmatrix}$$

$$J_g(x)= \nabla g = \left( \frac{\partial g}{\partial x_1},...,\frac{\partial g}{\partial x_n} \right)$$

And finally:

$$J_g(x)\cdot J_f(t)=\frac{\partial g}{\partial x_1}\frac{dx_1}{dt}+...+\frac{\partial g}{\partial x_n}\frac{dx_n}{dt}$$

That is the chain rule for this particular case.

Take also a look here: Derivation of the multivariate chain rule

The general theorem allow to find similar rules for any case by the Jacobian matrices $J_f$ and $J_g$.

• Thank you, that helps a lot actually. But I'm still unsure about the derivative of this function. We now know $g(f(x_0+h))=g(f(x_0))+J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)$ but to take the derivative we need to write $lim_{h\rightarrow 0}\frac{g(f(x_0+h)-g(f(x_0))}{h}=lim_{h\rightarrow 0} \frac{J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)}{h}$ I understand that little-oh can be neglected but what happens to the Jacobian matrices? – NKH Dec 10 '17 at 10:48
• In this case to understand deeply the rule it is necessary to generalize the context. The key point is that tha differential is a linear map, thus the composition is also a linear map i.e. a product of matrices /vectors (i.e.Jacobian, Gradient). – user Dec 10 '17 at 10:52
• Sorry there was a typo, I missed the h, now it's fixed! – user Dec 10 '17 at 10:54
• In the proof the onlydifficult part is to show that o(|h|)/h->0. – user Dec 10 '17 at 10:56

These days I've been looking for a rigurous proof of the multivariable chain rule and I've finally found one that I think is very easy to understand. I will leave it here (if nobody minds) for anybody searching for this that is not familiar with little-o notation, Jacobians and stuff like this. To understand this proof, all you need to know is the mean value theorem.

Let's say we have a function $$f(x,y)$$ and $$x = x(t), y = y(t)$$. Let's also take $$z(t) = f(x(t), y(t))$$ By definition, the derivative of z $$z'(t)$$ is

$$z'(t) = \lim_{\Delta t \to 0}{\frac {f(x(t+\Delta t),y(t+\Delta t)) - f(x,y)}{\Delta t}}$$.

$$Let \ \Delta x = x(t+\Delta t)-x(t),$$ $$\Delta y = y(t+\Delta t)-y(t)$$

Now I'll take the numerator of the fraction in the limit, and make a small change.

$$f(x(t+\Delta t), y(t+\Delta t)) - f(x,y) = f(x+\Delta x, y+\Delta y) - f(x,y)$$ $$= \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right] + \left[f(x+\Delta x, y) - f(x, y)\right]$$

I have just added and substracted $$f(x+\Delta x, y)$$. For some reason, I will invert the terms.

$$= \left[f(x+\Delta x, y) - f(x, y)\right] + \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right]$$.

Now, let's define 2 functions and I will name them g and h. First,

$$Let \ g(x) = f(x, y) \implies g'(x) = \frac {\partial f} {\partial x}$$.

Please note that y is constant here since g is a function of a single variable. Now, by the mean value theorem we have

$$\exists c_1 \in (x, x+\Delta x) \ so \ that$$ $$\frac {g(x+\Delta x) - g(x)} {\Delta x} = g'(c_1)$$ $$\Longleftrightarrow$$ $$f(x+\Delta x, y) - f(x, y) = f_x(c_1, y)\Delta x$$

Similarly, using the function $$h(y) = f(x + \Delta x, y) \implies h'(y) = \frac {\partial} {\partial y}f(x+\Delta x, y)$$

We will have by the same logic that

$$f(x+\Delta x, y + \Delta y) - f(x+\Delta x, y) = f_y(x + \Delta x, c_2)\Delta y, c_2 \in (y, y+\Delta y)$$

Notice that $$c_1$$ and $$c_2$$ are bounded with respect to $$\Delta x$$ and $$\Delta y$$ So as $$\Delta x \to 0, c_1 \to x$$ and as $$\Delta y \to 0, c_2 \to y$$. By our definition of $$\Delta x$$ and $$\Delta y$$, as $$\Delta t \to 0$$, both $$\Delta x$$ and $$\Delta y$$ $$\to 0$$. So, as $$\Delta t \to 0$$, $$c_1 \to x$$ and $$c_2 \to y$$.

The last step of the proof is to sum this all up, divide by $$\Delta t$$ and take the limit as $$\Delta t \to 0$$

$$f(x(t+\Delta t), y(t+\Delta t)) - f(x, y) = f_x(c_1, y)\Delta x + f_y(x+\Delta x, c_2)\Delta y$$ $$\lim_{\Delta t \to 0} \frac {f(x(t+\Delta t), y(t+\Delta t))}{\Delta t} = \lim_{\Delta t \to 0} f_x(c_1, y)\frac {\Delta x}{\Delta t} + f_y(x+\Delta x, c_2)\frac {\Delta y}{\Delta t} = f_x(x, y)x'(t) + f_y(x, y)y'(t) \ QED$$