# What's the derivative of an inner product with respect to the inner product matrix?

When you have an expression like this:

$$g \left ( \mathbf{X} \right ) = \mathbf{a}^T \mathbf{X} \mathbf{b},$$

where $$\mathbf{a} \in \mathbb{R}^d$$, $$\mathbf{b} \in \mathbb{R}^e$$ and $$\mathbf{X} \in \mathbb{R}^{d \times e}$$,

What's the matrix derivative (Jacobian?) of the expression with respect to $$\mathbf{X}$$, i.e. $$\frac{\partial g}{\partial \mathbf{X}} = \frac{\partial \left ( \mathbf{a}^T \mathbf{X} \mathbf{b} \right )}{\partial \mathbf{X}}$$?

My guess is that it is $$\mathbf{a} \mathbf{b}^T$$.

• You have correctly guessed the gradient. Out of curiosity, what do you plan to use for it?
– greg
Commented Jan 6, 2023 at 21:46
• In reality, there is another function inside the $\mathbf{X}$, I am exploring an optimization problem, where I have an operation like $\mathbb{1}^T \left ( A + B X \right ) \mathbb{1}$, and I want to derivate that expression with respect to $X$. Commented Jan 6, 2023 at 22:21

This map is linear in $$X$$, so it's derivative is the map itself: $$Dg(X)Y = g(Y) = a^T Y b$$ As a proof just write $$g(X+Y) - g(X) = a^T(X+Y)b - a^T X b = a^TYb$$ and apply the definition of the derivative of a multivariate function.
• Hi. The fact that $g$ is linear is clear, but I don't see what you mean by using the definition of the derivative of a multivariable function ? (I know the differentiability of a function) Commented Jan 6, 2023 at 21:46
• @coboy the defintion of the derivative of a map $f$ says that $f$ is differentiable in a point $x$ and the derivative is given by the linear map $A$ iff $f(X+Y) - f(X) = AY + o(|Y|)$ for some function $o$ which satisfies $o(|v|)/|v|\rightarrow 0$ when $v\rightarrow 0$. If you compare this with the equation I wrote down this means that $AY = a^tYb$ and $o$ is identically $0$ in this case. Commented Jan 6, 2023 at 21:52