4
$\begingroup$

Reading about differentiation of matrices seems to state that, given $f:\mathbb{R}^n\rightarrow \mathbb{R}$ and $g:\mathbb{R}^m\rightarrow \mathbb{R}^n$

$$\frac{d}{dx}(f(g)) = \frac{d(g^T)}{dx} \cdot \frac{\partial f}{\partial g}$$

(I think I might be wrong)

Why is the transposition needed? Moreover, does this mean that:

$\nabla f = \frac{d(f^T)}{dx} = (\frac{d(f)}{dx})^T$?

$\endgroup$
2
  • $\begingroup$ Product rule?${}$ $\endgroup$
    – markvs
    Jan 25 at 7:19
  • $\begingroup$ If $g$ was of the form $\mathbb{R}^n\to \mathbb{R}^m$, you wouldn't need to transpose. This is just so that the compositions make sense. $\endgroup$
    – Gabriel
    Jan 25 at 7:32

1 Answer 1

5
$\begingroup$

Note $f\circ g$ is a map from $\mathbb{R}^m$ to $\mathbb{R}$ so its derivative is a gradient vector. Depending on your convention, you would either write this gradient as a $m \times 1$ vector or a $1\times m$ vector. Appealing to the chain rule, your convention writes it as a $m\times 1$ vector :

$$D_x f\circ g=\underbrace{( D_xg^T)}_{m\times n}\underbrace{( D_g f)}_{n\times 1}.$$

The transpose is placed to be consistent within the convention you are working in. So your convention is set up such that putting a transpose on $g$ in $ D_xg^T$ is needed to consistently write it as the desired $m\times n$ result.

If it helps, you can refer to matrix derivative rules on wiki. Personally, I do matrix derivatives quick and dirty: I apply derivative rules treating everything as a scalar, and then rearrange the order of the matrices or throw in transposes as needed for the dimensions to work out, and usually this approach gets me the right answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.