# Multivariable chain rule with transposes

I am familiar with the multivariable chain rule. However, in the comments to this question, they have a version with matrix and transpose notation:

$$\frac{df(g,h)}{dx} = \frac{d(g(x)^T)}{dx} \frac{\partial f(g,h)}{\partial g} + \frac{d(h(x)^T)}{dx} \frac{\partial f(g,h)}{\partial h}$$

I'm not entirely sure on how this is equivalent to the standard notation. For instance, what does it mean to take the transpose of a function $$g(x)$$?

## 1 Answer

I'm not entirely sure on how this is equivalent to the standard notation. For instance, what does it mean to take the transpose of a function g(x)?

It is sensible when the image of the function is itself a vector.

The general form for the chain rule acting on function with two vector arguments of a vector is:$$\dfrac{\mathrm df(\vec g,\vec{~}\hspace{-1ex}h)}{\mathrm d\vec x}=\dfrac{\mathrm d(\vec g(\vec x)^\top)}{\mathrm d\vec x}\dfrac{\partial f(\vec g,\vec{~}\hspace{-1ex}h)}{\partial\vec g}+\dfrac{\mathrm d(\vec{~}\hspace{-1ex}h(\vec x)^\top)}{\mathrm d\vec x}\dfrac{\partial f(\vec g,\vec{~}\hspace{-1ex}h)}{\partial\vec{~}\hspace{-1ex}h}$$

Applying this to $$f(\vec g,\vec{~}\hspace{-1ex}h)=\vec g^\top\vec{~}\hspace{-1ex}h$$ where $$\vec g=\vec x$$ and $$\vec{~}\hspace{-1ex} h=\mathrm A\vec x$$, and taking things slowly:

\begin{align}\dfrac{\mathrm d (\vec g^\top\vec{~}\hspace{-1ex}h) }{\mathrm d\vec x}&=\dfrac{\mathrm d(\vec g^\top)}{\mathrm d\vec x~~}\dfrac{\partial(\vec g^\top\vec{~}\hspace{-1ex} h) }{\partial\vec g}+\dfrac{\mathrm d (\vec{~}\hspace{-1ex}h^\top)}{\mathrm d\vec x}\dfrac{\partial (\vec g^\top\vec{~}\hspace{-1ex}h)}{\partial h}\\[1ex]&=\dfrac{\mathrm d(\vec x^\top)}{\mathrm d\vec x~~}\dfrac{\partial(\vec g^\top\vec{~}\hspace{-1ex} h) }{\partial\vec g}+\dfrac{\mathrm d (\mathrm A\vec x)^\top}{\mathrm d\vec x}\dfrac{\partial (\vec g^\top\vec{~}\hspace{-1ex}h)}{\partial h}\\[1ex]&= \dfrac{\partial(\vec g^\top\vec{~}\hspace{-1ex} h) }{\partial\vec g}+\dfrac{\mathrm d (\vec x^\top\mathrm A^\top)}{\mathrm d\vec x}\dfrac{\partial (\vec{~}\hspace{-1ex}h^\top\vec g)}{\partial h}\\[1ex]&=\vec{~}\hspace{-1ex}h+\mathrm A^\top\vec g\\[1ex]&=\mathrm A\vec x+\mathrm A^\top\vec x\\[1ex]&=(\mathrm A+\mathrm A^\top)~\vec x\end{align}

Also using $$\dfrac{\partial (\vec v^\top\mathbf c)}{\partial \vec v}=\mathbf c$$ and $$(\vec u^\top\vec v)=(\vec v^\top\vec u)$$