Another question on reciprocals of partial derivatives Consider the function $f(\mathbf{x},g(\mathbf{x}))$, where $f:\mathbb{R}^n\times\mathbb{R}^n\to \mathbb{R}$, $\mathbf{x}\in\mathbb{R}^n$ and $g:\mathbb{R}^n\to\mathbb{R}^n$.
I want to take the gradient $\nabla_{\mathbf{x}}f(\mathbf{x},g(\mathbf{x}))$, which I think should give me this...
$$
\nabla_{\mathbf{x}}f(\mathbf{x},g(\mathbf{x})) = \left[\frac{\partial f}{\partial x_1} + \frac{\partial f}{\partial g}\frac{\partial g}{\partial x_1},\cdots,\frac{\partial f}{\partial x_n} + \frac{\partial f}{\partial g}\frac{\partial g}{\partial x_n}\right]^{T}
$$
Finding the derivatives $\frac{\partial g}{\partial x_i}$ is a little tricky.  Naively I thought I could do something like
$$
\frac{\partial g}{\partial x_i} = \frac{\partial g}{\partial f}\frac{\partial f}{\partial x_i}
$$
under the assumption that $\frac{\partial g}{\partial f} = \left(\frac{\partial f}{\partial g}\right)^{-1}$, yet substituting such an expression into the equation for the gradient would give me the following form
$$
\nabla_{\mathbf{x}}f(\mathbf{x},g(\mathbf{x})) = \left[\cdots,2\frac{\partial f}{\partial x_i},\cdots\right]
$$
Clearly the reciprocal argument used to find $\frac{\partial g}{\partial f}$ has been abused here, but I'm not clear why.  I was under the impression that such an argument could be used as long as the same variables are being held constant, and I think I am doing that.  I guess the remaining weak spot is that I have fundamentally misunderstood the application if inverse function theorem in this context?
 A: Define $h(x) = f(x,g(x))$ and let suppose that both $f,g$ are continuously differentiable. $h$ is defined on $\mathbb R^n$ and takes values in $\mathbb R$.
We can write
$$h(x) = (f \circ k)(x)$$ where $k(x) = (x, g(x))$ is a map from $\mathbb R^n$ to $\mathbb R^n \times \mathbb R^n$. The Fréchet or total derivative of $k$ at $x$ is given by
$$k^\prime(x)(r) = (r, g^\prime(x)(r)).$$ For $x \in \mathbb R^n$ given, $k^\prime(x)$ is a linear map from $\mathbb R^n$ to $\mathbb R^n \times \mathbb R^n$. Knowing that the derivative of $f$ at $(x_1,x_2)$ is
$$f^\prime(x_1,x_2)(r,s) = \left(\frac{\partial f}{\partial x_1}(x_1,x_2)\right)(r) + \left(\frac{\partial f}{\partial x_2}(x_1,x_2)\right)(s) $$ we get applying the chain rule:
$$\begin{aligned}h^\prime(x)(r) &= \left(\frac{\partial f}{\partial x_1}(x,g(x))\right)(r) + \left(\frac{\partial f}{\partial x_2}(x,g(x))\right)(g^\prime(x)(r)\\
&=\left(\frac{\partial f}{\partial x_1}(x,g(x)) + \left(\frac{\partial f}{\partial x_2}(x,g(x)) \circ g^\prime(x)\right)\right)(r)
\end{aligned} $$ For $x \in \mathbb R^n$ given, $h^\prime(x)$ is a linear map from $\mathbb R^n$ to $\mathbb R$. In term of gradient notation we would get:
$$\nabla h(x) = (\partial_1 f(x,g(x)))^T+ (Dg(x))^T(\partial_2 f(x,g(x)))^T$$ where $Dg(x)$ is the Jacobian of $g$ at $x$.
Note 1: as always with multivariate functions and derivatives, it is critical to understand to which spaces belong all the written maps.
Note 2: on my side, I much prefer total derivative or Fréchet derivative notations that are more general than gradient, Jacobian, and so on. But that's a question of test!
