Derivative of quadratic matrix form with respect to the matrix

Question

Suppose we have the following quadratic form

$$ f(M)=x^TMx $$ where $f: \mathbb{R}^{n \times m} \rightarrow \mathbb{R}$, and $x \in \mathbb{R}^n$. As it is obvious, this function is linear in $M$. What is the derivative of $f(M)$ with respect to $M$. I think it is $xx^T$ but why?

Answer 1

$$ f(M)=\sum_{i,j}x_iM_{i,j}x_j. $$ Therefore $$ \frac{\partial f(M)}{\partial M_{i,j}} = x_ix_j = (xx^T)_{i,j}. $$

Answer 2

The strategy is to write the expression as a scalar using index notation, take the derivative, and re-write in matrix form.

Note that to write the function as a summation of matrices we have to write just one scalar as a matrix multiplication because the function is scalar: $$ f(M)= [x^TMx]_{11}= \sum_i x_{i1}[Mx]_{i1} $$ At first summation, we have $x_{i1}$ because we have $x^T$ as the first term.

Now expand $[Mx]_{i1}$ $$ f(M)= [x^TMx]_{11}= \sum_i x_{i1}\sum_j M_{ij}x_{j1}=\sum_i \sum_j x_{i1} M_{ij}x_{j1} $$

Now take the derivative with respect to $M_{ij}$

$$ \frac{\partial f(M)}{\partial M_{ij}}=\sum_i \sum_j x_{i1}x_{j1} $$

Looking at the indices, we can see that

$$ \sum_i \sum_j x_{i1}x_{j1}=\sum_i \sum_j x_{j1}x_{i1}=[xx^T]_{ji} $$

Therefore,

$$ \frac{\partial f(M)}{\partial M_{ij}}=xx^T $$