How to take derivative of a function $F \circ G$ when $F: \mathbb R \times \mathbb R \to \mathbb R$ and $G: \mathbb R \to \mathbb R \times \mathbb R$

Question

The only rigorous familiarity I have with calculus involves single variable real functions of the form $\mathbb R \to \mathbb R$. In a recent problem I worked on, I defined the following functions:

\begin{align} &(1) \quad f:\mathbb R \times \mathbb R \to \mathbb R, \text{ where }(x,y)\mapsto\sqrt{(x-x_1)^2+(y-y_1)^2} \\&(2)\quad g: \mathbb R \to \mathbb R \times \mathbb R, \text{ where }\theta\mapsto \left(\cos(\theta)\cdot r_0+x_0,\sin(\theta)\cdot r_0+y_0\right) \end{align}

After the definition, I carried out the composition $f \circ g$. I am interested in finding the derivative of this function for an optimization-related pursuit.

If $f$ and $g$ were both defined as $\mathbb R \to \mathbb R$, I would carry out the chain rule as is traditionally taught: $(f\circ g)(x)=f'\left(g(x)\right)g'(x)$.

Clearly, my functions do not satisfy the relevant criteria for the above case...and I am not really sure what to do.

When we write the composition in terms of $\theta$, we get:

$$\left(f\circ g\right)(\theta)=\sqrt{\left(\cos(\theta)\cdot r_0+x_0-x_1\right)^2+\left(\sin(\theta)\cdot r_0+y_0-y_1\right)^2} \quad (*)$$

Suppose, for a second, that someone told me to take the derivative of the function:

$$T: \mathbb R \to \mathbb R, \text{ where } \theta \mapsto\sqrt{\left(\cos(\theta)\cdot r_0+x_0-x_1\right)^2+\left(\sin(\theta)\cdot r_0+y_0-y_1\right)^2}$$

I would say, "No problem". However, in the case of $(*)$, if we interpret the function as a composition, then I am not really sure what it means to take the derivative of a functions of the form $\mathbb R \times \mathbb R \to \mathbb R$ or $\mathbb R \to \mathbb R \times \mathbb R$.

When providing the readers with my specific example, it is clear that $f \circ g = T$, so, presumably, $(f\circ g)'=T'$. Instead, suppose I only knew the most general things about my $f$ and $g$ functions: domain and range. i.e. suppose I knew nothing about the specific mapping rule.

Is it a sufficient proof to say that any composition of the form $f_1 \circ f_2 \circ \cdots \circ f_n$ that has domain $\mathbb R$ and codomain $\mathbb R$ must necessarily have an equivalent representation as a function $g$ of the form $g: \mathbb R \to \mathbb R$, where $\theta \mapsto f_1 \circ f_2 \circ \cdots \circ f_n (\theta)$?

In which case, despite my knowledge being restricted to single variable calculus of the form $\mathbb R \to \mathbb R$, I can find the derivative of $f_1 \circ f_2 \circ \cdots \circ f_n$ simply by taking the derivative of $g$? That is to say that the domain of $f_2, f_3, \cdots, f_n$ and the codmains of $f_1, f_2, \cdots , f_{n-1}$ can be literally anything and it is still business as usual (within the framework of single variable $\mathbb R \to \mathbb R$ function calculus), right?

Answer 1

You need to look up the Frechet derivative and multivariable chain rule. Briefly, if $f:\Bbb{R}^n\to\Bbb{R}^m$ and $a\in \Bbb{R}^n$, then the derivative (when it exists) at $a$ is a linear transformation $Df_a:\Bbb{R}^n\to\Bbb{R}^m$ (the Frechet/total derivative of $f$ at $a$). As you may hopefully know from linear algebra, every linear transformation can be assigned a matrix once you choose bases on the domain and target vector space. In our case, use the usual standard bases. This gives the matrix representation $f'(a)$ of $Df_a$; this is also known as the Jacobian matrix. It's entries are the partial derivatives of the function $f$.

The chain rule asserts (under the "obvious" hypothesis that the composition makes sense, and the functions are differentiable at the correct points), $D(f\circ g)_a=Df_{g(a)}\circ Dg_a$, or as an equation relating matrices, $(f\circ g)'(a)=f'(g(a))\cdot g'(a)$, where the $\cdot$ is matrix multiplication.

Here is an answer where I go much more in detail (the definition of differentiability in higher dimensions, and the chain rule are the most important basic steps in differential calculus, so it's well worth it to study it very carefully).

Regarding your final few paragraphs, as long as $f_n$ has domain (an open subset of) $\Bbb{R}$ and $f_1$ has target space $\Bbb{R}$, it doesn't matter what anything else is. The resulting composition (assuming all the compositions line up and are well-defined) is $f_1\circ \cdots \circ f_n:\Bbb{R}\to\Bbb{R}$. If you have an explicit formula for this function, you can decide whether or not it is differentiable at a given point, so you're reduced to the basic single-variable case.

Just to be explicit, let $E=\{\ddot{\smile}, \ddot{\frown}\}$ be this wacky two-element set, let $f_2:\Bbb{R}\to E$ be $f_2(x)=\ddot{\smile}$ and let $f_1:E\to\Bbb{R}$ be defined as say $f_1(\ddot{\smile})=1$, $f_1(\ddot{\frown})=0$. Then, $f_1\circ f_2$ is the constant function $x\mapsto 1$, so it is certainly differentiable. However, it obviously doesn't make sense to talk about differentiability of $f_1$ or of $f_2$. So, bottom line is if you know everything about the fully composed function $f_1\circ f_2$ (or more generally $f_1\circ \cdots \circ f_n$), then you can forget about the fact that it was obtained via composition, and it is indeed back-to-basics single-variable calculus as usual (though of course, the multivariable chain rule, when it applies, is an extremely powerful tool).

Answer 2

Yes, there is a generalized chain rule. It is based on a generalized definition of the derivative.

The generalized chain rule

Consider a function $F\colon \mathbb R^m\to \mathbb R^n$ (more generally, we can consider a function from an open subset of a normed affine space - i.e. an affine space with a norm on the translation space - to a normed affine space, the formulas remain the same). Its directional derivative (if existent) is the function \begin{align}DF\colon \mathbb R^m\times\mathbb R^m&\to\mathbb R^n\\(x,v)&\mapsto\lim_{\epsilon\to 0}\frac{F(x+\epsilon v)-F(x)}{\epsilon} \end{align} and the directional derivative satisfies the chain rule:$$D(G\circ F)(x,v)=DG(F(x),DF(x,v))$$Note that for each $x\in \mathbb R^m$, the function $DF_x\colon\mathbb R^m\to\mathbb R^n$ defined by$$DF_x(v)=DF(x,v)$$is linear, i.e. $DF_x\in L(\mathbb R^m,\mathbb R^n)$, and we can restate the chain rule as follows:$$D(G\circ F)_x=DG_{F(x)}\circ DF_x$$

Why it really is a generalization

In the case $m=n=1$ we have the natural isomorphism \begin{align}L(\mathbb R,\mathbb R)&\to\mathbb R\\A&\mapsto A(1)\end{align} and given a differentiable function $f\colon \mathbb R\to\mathbb R$ we have $f'(x)=Df(x,1)$.

How this applies to your problem

By our definition of the directional derivative, we have $$\partial_i F_j(x)=DF(x,e_i)_j$$ for a function $F\colon \mathbb R^m\to \mathbb R^n$, where $e_1,\ldots,e_m$ is the Standard basis of $\mathbb R^m$. You can use this together with the chain rule and linearity to obtain the desired result: $$(G\circ F)'(x)=\sum_{i=1}^2\partial_iG(F(x))F_i{}'(x)$$