I would like to take the derivative of the following expression w.r.t. the matrix $A \in \mathbb{R}^{m \times n}$, i.e.,

$$ \frac{\partial \big( \sum_{i=1}^m (Ax)_i \big)}{\partial A}, $$

where $x \in \mathbb{R}^n$. The second answer here gives the derivative of the matrix-vector product w.r.t. the matrix, but, I wasn't sure how it changes with the summation? Though I think that it should work out cleanly since derivative and summation are linear operators and can be interchanged? I am not sure about how the indices would change so any advice regarding that would be much appreciated.

  • $\begingroup$ Over what set of values does $i$ run? How if at all does $Ax$ depend on it? $\endgroup$ – J.G. Jul 22 '19 at 19:59
  • $\begingroup$ @J.G. apologies, have updated it. $\endgroup$ – user11726509 Jul 22 '19 at 20:02

First note that $\sum_i (Ax)_i = {\bf 1}^T A x$ where $\bf 1$ is a vector of ones. Moreover, $\frac{\partial}{\partial A}y^TAx = yx^T$, so

$$\frac{\partial (\sum_i (Ax)_i)}{\partial A} = {\bf 1}\cdot x^T =\begin{pmatrix} x_1&x_2&\cdots&x_n\\ x_1&x_2&\cdots&x_n\\ \vdots\\ x_1&x_2&\cdots&x_n\\ \end{pmatrix}$$


Thanks to the summation over $i$, you're differentiating a quantity with no indices with respect to a matrix, so obtain a matrix. Its components are$$\frac{\sum_{il}\partial(A_{il}x_l)}{\partial A_{jk}}=\sum_{il}\delta_i^j\delta_k^lx_l=x_k.$$This agrees with @Hyperplane's result $1\cdot x^T$.

  • $\begingroup$ @user11726509 I forgot about your $\sum_i$, but my edit addresses this. $\endgroup$ – J.G. Jul 22 '19 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.