Linearity of derivative sanity check

Question

Currently I'm reading about the formalization of the derivative as a linear operator and more specifically reinterpreting the chain rule. Given $f_1, f_2:\mathbb{R}^n\rightarrow\mathbb{R}^p$ and $g(x_1,x_2):\mathbb{R}^p\times\mathbb{R}^p\rightarrow\mathbb{R}^p$ where $(x_1,x_2)\mapsto \lambda x_1+x_2$, lets take the derivative of $g(f_1, f_2)$ or in other words $\lambda f_1+ f_2$. I want to apply the chain rule without using Jacobian matrices or partial derivatives too directly, so I used linear operator form: $$D(g\circ f)(a)=Dg(f(a))\circ D(f(a))$$ or $$D(g\circ f)=((Dg)\circ f)\circ Df$$

Here we know from the limit definition that $Dg=g=\lambda x_1 +x_2$ is equal to itself. If $f=(f_1,f_2)$, I get $(Dg)\circ f=\lambda f_1+f_2$ as the first term and so the total derivative is $(\lambda f_1+f_2)\circ (Df_1, Df_2)$ which is totally incorrect. But we know $Df=(Df_1, Df_2)$, so must $(Dg)\circ f$ be $g$ instead?

This should be simple but I am probably misinterpreting the chain rule here.

Answer 1

Your issues stem from the fact that you're abusing the notation for composition (which is also evident when you write things like $g(f_1,f_2)$ as shorthand for $g\circ (f_1, f_2)$). The equality \begin{align} D(g\circ f)&=((Dg)\circ f)\circ Df \end{align} is strictly speaking false. Suppose we have three (say real, normed, finite-dimensional) vector spaces $X,Y,Z$ (which you may like to assume as $\Bbb{R}^{n_1}, \Bbb{R}^{n_2},\Bbb{R}^{n_3}$ if you wish). Suppose we have two maps $f:X\to Y$ and $g:Y\to Z$, then we can form the composition $g\circ f:X\to Z$.

Now, $Df$ is a function $X\to \mathcal{L}(X,Y)$, and $Dg$ is a function $Y\to \mathcal{L}(Y,Z)$. Therefore, it should make sense to you that the composition $f\circ Df$ isn't even well-defined.

The chain rule is properly written in the first equation you wrote: for every $a\in X$, \begin{align} D(g\circ f)_a&=Dg_{f(a)}\circ Df_a \end{align} (I put the point $a$ in subscript position simply for easier reading). On the LHS we have an element of $\mathcal{L}(X,Z)$, and on the right we have a composition of elements in $\mathcal{L}(Y,Z)$ and $\mathcal{L}(X,Y)$, so everything makes sense.

Now, even in the special case that $g:Y\to Z$ is a linear transformation, i.e $g\in\mathcal{L}(Y,Z)$, saying $Dg=g$ is a completely nonsensical statement. $Dg:Y\to\mathcal{L}(Y,Z)$ and $g:Y\to Z$ have completely different target spaces, so they certainly cannot be equal as functions. What is true though is that when $g$ is linear, $Dg:Y\to\mathcal{L}(Y,Z)$ is a constant function, whose value at any $y\in Y$ is $Dg_y=g$; i.e the linear transformation $Dg_y(\cdot)$ is equal to the linear transformation $g(\cdot)$ (a proper equality of elements in $\mathcal{L}(Y,Z)$).

Once again: the equation $Dg=g$ is an abuse of notation, so only use it if you absolutely know what you're talking about. For example, for a function $f:\Bbb{R}\to\Bbb{R}$, if we write $f=1$, then I'm sure everyone will have no trouble understanding that we mean that for every $x\in\Bbb{R}$, $f(x)=1$; i.e $f$ is the constant function whose value is always the number $1$. But then again, technically, $f=1$ is an abuse of notation, since a function cannot equal a real number. When the domains and target spaces get more complicated, I suggest you to be extra cautious about where exactly one is abusing notation.

---

To address the application of the chain rule here directly for proving linearity of $D$, take a look at How to use chain rule properly/rigorously with functions which doesn't have explicit formulas? where I explain how exactly the computation works.