# Proof of multivariable chain rule

I'm working with a proof of the multivariable chain rule $$\displaystyle{\frac{d}{dt}g(t)=\frac{df}{dx_1}\frac{dx_1}{dt}+\frac{df}{dx_2}\frac{dx_2}{dt}}$$ for $$g(t)=f(x_1(t),x_2(t))$$, but I have a hard time understanding two important steps of this proof.

The proof includes the function $$\displaystyle{\Delta_i(h)=x_i(t+h)-x_i(t)}$$ for $$\displaystyle{i=1,2, \bar{\Delta}=(\Delta_1(h),\Delta_2(h)) \Rightarrow \lim_{h\rightarrow0}\frac{\Delta_i}{h}=x^{'}_i}$$. It says that

$$\frac{g(t+h)-g(t)}{h}=\frac{f(\bar{x}(t+h))-f(\bar{x}(t))}{h}=\frac{f(\bar{x}(t)-\bar{\Delta(h))}-f(\bar{x}(t))}{h}$$

which I understand, but the next step is the to state that $$f$$ is differentiable and then let the previous equation be equal to

$$=f^{'}_1(\bar{x}(t))\cdot\Delta_1(h)+f^{'}_2(\bar{x}(t))\cdot\Delta_2(h)+o(\vert\vert\bar{\Delta}\vert\vert)$$

and this step I do not understand. I think there might be missing some limit-notation? But even with the limit notation I'm still not sure as to how it becomes a partial derivative multiplied with $$\Delta_i$$.

Afterwards they let $$h\rightarrow0$$ to get

$$=f^{'}_1(\bar{x}(t))\cdot x_1^{'}(t)+f^{'}_2(\bar{x}(t))\cdot x_2^{'}(t)$$

Again I am very confused as to possibly missing limit notations.

Does anyone know this version of the proof of the chain rule (besides these two steps, I find it the easiest version to understand), or understand these steps?

Here are pictures of the notes:

Theorem: Multivariable chain rule

Proof of theorem

• See this page regarding the "little-oh notation" : en.wikipedia.org/wiki/Big_O_notation . Although it says Big-Oh, there is a section on little-oh on this page as well. Dec 10, 2017 at 9:31
• Actually, I see the logic behind the little-oh - it's more the products of the partial derivaties and lambda-functions that I can not see how comes up :)
– NKH
Dec 10, 2017 at 9:39
• Well first,this is not really a proof but an informal argument. You won't see a real proof of either single or multivariate chain rules until you take real analysis. Could you upload a picture of this for me so I can see the entire thing? Also when you say lambda do you mean delta? Dec 10, 2017 at 9:56
• Of course, sorry, I meant delta. I have now edited the question and uploaded a picture of the notes I have.
– NKH
Dec 10, 2017 at 10:15
• Consider that you are dealing wih: $$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$ $$g: \mathbb{R^n} \to \mathbb{R}$$ $$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$ To prove you only have to use the definition of differtial showing that the composition is also a differential.
– user
Dec 10, 2017 at 10:43

These days I've been looking for a rigurous proof of the multivariable chain rule and I've finally found one that I think is very easy to understand. I will leave it here (if nobody minds) for anybody searching for this that is not familiar with little-o notation, Jacobians and stuff like this. To understand this proof, all you need to know is the mean value theorem.

Let's say we have a function $$f(x,y)$$ and $$x = x(t), y = y(t)$$. Let's also take $$z(t) = f(x(t), y(t))$$ By definition, the derivative of z $$z'(t)$$ is

$$z'(t) = \lim_{\Delta t \to 0}{\frac {f(x(t+\Delta t),y(t+\Delta t)) - f(x,y)}{\Delta t}}$$.

$$Let \ \Delta x = x(t+\Delta t)-x(t),$$ $$\Delta y = y(t+\Delta t)-y(t)$$

Now I'll take the numerator of the fraction in the limit, and make a small change.

$$f(x(t+\Delta t), y(t+\Delta t)) - f(x,y) = f(x+\Delta x, y+\Delta y) - f(x,y)$$ $$= \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right] + \left[f(x+\Delta x, y) - f(x, y)\right]$$

I have just added and substracted $$f(x+\Delta x, y)$$. For some reason, I will invert the terms.

$$= \left[f(x+\Delta x, y) - f(x, y)\right] + \left[f(x+\Delta x, y+\Delta y) - f(x+\Delta x, y)\right]$$.

Now, let's define 2 functions and I will name them g and h. First,

$$Let \ g(x) = f(x, y) \implies g'(x) = \frac {\partial f} {\partial x}$$.

Please note that y is constant here since g is a function of a single variable. Now, by the mean value theorem we have

$$\exists c_1 \in (x, x+\Delta x) \ so \ that$$ $$\frac {g(x+\Delta x) - g(x)} {\Delta x} = g'(c_1)$$ $$\Longleftrightarrow$$ $$f(x+\Delta x, y) - f(x, y) = f_x(c_1, y)\Delta x$$

Similarly, using the function $$h(y) = f(x + \Delta x, y) \implies h'(y) = \frac {\partial} {\partial y}f(x+\Delta x, y)$$

We will have by the same logic that

$$f(x+\Delta x, y + \Delta y) - f(x+\Delta x, y) = f_y(x + \Delta x, c_2)\Delta y, c_2 \in (y, y+\Delta y)$$

Notice that $$c_1$$ and $$c_2$$ are bounded with respect to $$\Delta x$$ and $$\Delta y$$ So as $$\Delta x \to 0, c_1 \to x$$ and as $$\Delta y \to 0, c_2 \to y$$. By our definition of $$\Delta x$$ and $$\Delta y$$, as $$\Delta t \to 0$$, both $$\Delta x$$ and $$\Delta y$$ $$\to 0$$. So, as $$\Delta t \to 0$$, $$c_1 \to x$$ and $$c_2 \to y$$.

The last step of the proof is to sum this all up, divide by $$\Delta t$$ and take the limit as $$\Delta t \to 0$$

$$f(x(t+\Delta t), y(t+\Delta t)) - f(x, y) = f_x(c_1, y)\Delta x + f_y(x+\Delta x, c_2)\Delta y$$ $$\lim_{\Delta t \to 0} \frac {f(x(t+\Delta t), y(t+\Delta t))}{\Delta t} = \lim_{\Delta t \to 0} f_x(c_1, y)\frac {\Delta x}{\Delta t} + f_y(x+\Delta x, c_2)\frac {\Delta y}{\Delta t} = f_x(x, y)x'(t) + f_y(x, y)y'(t) \ QED$$

Edit: After a long time I've realised that this proof assumes that $$f$$ has partial derivatives defined on intervals around the point $$(x, y)$$ and they are continuous at the point. This is a sufficient condition for the function to be ($$\mathbb{R}^2$$-)differentiable at $$(x, y)$$, but it's not equivalent. Yet, the multivariable chain rule works for the function being just differentiable at that point. So for a general proof, one should first understand little-o notation as in the other answers.

• Can you use this to prove for more than 2 variables? Apr 15, 2020 at 16:01
• Yes, this generalises fairly easily to the multivariable chain rule: for a function $f({\bf x})$ where ${\bf x}$ is an n-dimensional vector, $\frac{d}{dt} f({\bf x}(t)) = \nabla f({\bf x}(t)) \cdot {\bf x}'(t)$ Apr 15, 2020 at 19:38
• Very straightforward, yet rigorous proof. One small modification I would do is defining $c_i$ as $c_i = \theta_ix + (1-\theta_i)\Delta x$ for some $\theta_i\in(0,1)$. This way you make sure that the negative case is automatically covered. Also, curious whether this proof also covers the cases where e.g. $\Delta x$ rapidly takes value of $0$ close to the limit point, such as when $x(t) = x^2\sin(1/x)$. Aug 27, 2020 at 20:28
• @Viktorio El Hakim Both issues can be covered (I think) by letting c_1 = \left\{\begin{align*}&x + (1-\theta_1)\Delta x & \Delta x \ne 0 \\ &x & \Delta x = 0 \end{align*}\right. This way, $g(x+\Delta x) - g(x) = g'(c_1)\Delta x$ is still true, MVT covering the first branch and $0 = 0$ the second one, and $\lim_{t\to 0} c_1(t) = x$. I'm not sure if there are any flaws to this, thanks for pointing those out. Aug 28, 2020 at 14:42

Multivariable chain rule descends from the theorem of composite function for function of several variables which states in general that if:

f and g are differentiable in $$x_0$$ and $$y_0=f(x_0)$$, that is:

$$f(x_0+h)=f(x_0)+J_f(x_0)\cdot h+o(|h|)$$

$$g(y_0+k)=g(y_0)+J_g(y_0)\cdot k+o(|k|)$$

The composite function $$g \circ f$$ is also differentiable in $$x_0$$ and:

$$g(f(x_0+h))=g(f(x_0))+J_g(y_0)\cdot J_f(x_0)\cdot h+o(|h|)$$

NOTE

For the proof it is convenient to write:

$$o(|h|)=|h|\cdot \omega_f(h)$$ with $$\omega_f(h) \to 0$$

$$o(|k|)=|k|\cdot \omega_g(k)$$ with $$\omega_g(k) \to 0$$.

In the special case of:

$$f: \mathbb{R} \to \mathbb{R^n} \quad t \to f(x_1(t),x_2(t),...,x_n(t))$$

$$g: \mathbb{R^n} \to \mathbb{R}$$

$$\phi=g\circ f: \mathbb{R} \to \mathbb{R}$$

we have

$$J_f(t)= \begin{bmatrix} \frac{dx_1}{d t} \\ .\\\frac{dx_n}{d t} \end{bmatrix} = \begin{bmatrix} x_1' \\ .\\ x_n' \end{bmatrix}$$

$$J_g(x)= \nabla g = \left( \frac{\partial g}{\partial x_1},...,\frac{\partial g}{\partial x_n} \right)$$

And finally:

$$J_g(x)\cdot J_f(t)=\frac{\partial g}{\partial x_1}\frac{dx_1}{dt}+...+\frac{\partial g}{\partial x_n}\frac{dx_n}{dt}$$

That is the chain rule for this particular case.

Take also a look here: Derivation of the multivariate chain rule

The general theorem allow to find similar rules for any case by the Jacobian matrices $$J_f$$ and $$J_g$$.

• Thank you, that helps a lot actually. But I'm still unsure about the derivative of this function. We now know $g(f(x_0+h))=g(f(x_0))+J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)$ but to take the derivative we need to write $lim_{h\rightarrow 0}\frac{g(f(x_0+h)-g(f(x_0))}{h}=lim_{h\rightarrow 0} \frac{J_g(x_0)\cdot J_f(x_0)+o(\vert h\vert)}{h}$ I understand that little-oh can be neglected but what happens to the Jacobian matrices?
– NKH
Dec 10, 2017 at 10:48
• In this case to understand deeply the rule it is necessary to generalize the context. The key point is that tha differential is a linear map, thus the composition is also a linear map i.e. a product of matrices /vectors (i.e.Jacobian, Gradient).
– user
Dec 10, 2017 at 10:52
• Sorry there was a typo, I missed the h, now it's fixed!
– user
Dec 10, 2017 at 10:54
• In the proof the onlydifficult part is to show that o(|h|)/h->0.
– user
Dec 10, 2017 at 10:56