0
$\begingroup$

When you have an expression like this:

$$g \left ( \mathbf{X} \right ) = \mathbf{a}^T \mathbf{X} \mathbf{b},$$

where $\mathbf{a} \in \mathbb{R}^d$, $\mathbf{b} \in \mathbb{R}^e$ and $\mathbf{X} \in \mathbb{R}^{d \times e}$,

What's the matrix derivative (Jacobian?) of the expression with respect to $\mathbf{X}$, i.e. $\frac{\partial g}{\partial \mathbf{X}} = \frac{\partial \left ( \mathbf{a}^T \mathbf{X} \mathbf{b} \right )}{\partial \mathbf{X}}$?

My guess is that it is $\mathbf{a} \mathbf{b}^T$.

$\endgroup$
2
  • $\begingroup$ You have correctly guessed the gradient. Out of curiosity, what do you plan to use for it? $\endgroup$
    – greg
    Commented Jan 6, 2023 at 21:46
  • $\begingroup$ In reality, there is another function inside the $\mathbf{X}$, I am exploring an optimization problem, where I have an operation like $\mathbb{1}^T \left ( A + B X \right ) \mathbb{1}$, and I want to derivate that expression with respect to $X$. $\endgroup$
    – cserpell
    Commented Jan 6, 2023 at 22:21

2 Answers 2

3
$\begingroup$

This map is linear in $X$, so it's derivative is the map itself: $$Dg(X)Y = g(Y) = a^T Y b$$ As a proof just write $$g(X+Y) - g(X) = a^T(X+Y)b - a^T X b = a^TYb$$ and apply the definition of the derivative of a multivariate function.

$\endgroup$
3
  • $\begingroup$ Hi. The fact that $g$ is linear is clear, but I don't see what you mean by using the definition of the derivative of a multivariable function ? (I know the differentiability of a function) $\endgroup$
    – G2MWF
    Commented Jan 6, 2023 at 21:46
  • 2
    $\begingroup$ @coboy the defintion of the derivative of a map $f$ says that $f$ is differentiable in a point $x$ and the derivative is given by the linear map $A$ iff $f(X+Y) - f(X) = AY + o(|Y|)$ for some function $o$ which satisfies $o(|v|)/|v|\rightarrow 0$ when $v\rightarrow 0$. If you compare this with the equation I wrote down this means that $AY = a^tYb$ and $o$ is identically $0$ in this case. $\endgroup$
    – Thomas
    Commented Jan 6, 2023 at 21:52
  • $\begingroup$ Very clear. Thank you ! $\endgroup$
    – G2MWF
    Commented Jan 6, 2023 at 22:05
1
$\begingroup$

You can find a ton of matrix derivative identities here: https://en.wikipedia.org/wiki/Matrix_calculus

For your question, look in the scalar-by-matrix derivative formulas section.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .