# Derivative of quadratic matrix form with respect to the matrix

Suppose we have the following quadratic form

$$f(M)=x^TMx$$ where $$f: \mathbb{R}^{n \times m} \rightarrow \mathbb{R}$$, and $$x \in \mathbb{R}^n$$. As it is obvious, this function is linear in $$M$$. What is the derivative of $$f(M)$$ with respect to $$M$$. I think it is $$xx^T$$ but why?

• Since $f$ is linear, its differential is equal to $f$... Nov 1, 2018 at 0:35
• Can you write down the calculation for me? Nov 1, 2018 at 0:38
• Related Apr 26, 2021 at 1:48

$$f(M)=\sum_{i,j}x_iM_{i,j}x_j.$$ Therefore $$\frac{\partial f(M)}{\partial M_{i,j}} = x_ix_j = (xx^T)_{i,j}.$$

• Could you explain the last step? Nov 1, 2018 at 0:40
• What is $xx^T$ for you? Nov 1, 2018 at 0:41

The strategy is to write the expression as a scalar using index notation, take the derivative, and re-write in matrix form.

Note that to write the function as a summation of matrices we have to write just one scalar as a matrix multiplication because the function is scalar: $$f(M)= [x^TMx]_{11}= \sum_i x_{i1}[Mx]_{i1}$$ At first summation, we have $$x_{i1}$$ because we have $$x^T$$ as the first term.

Now expand $$[Mx]_{i1}$$ $$f(M)= [x^TMx]_{11}= \sum_i x_{i1}\sum_j M_{ij}x_{j1}=\sum_i \sum_j x_{i1} M_{ij}x_{j1}$$

Now take the derivative with respect to $$M_{ij}$$

$$\frac{\partial f(M)}{\partial M_{ij}}=\sum_i \sum_j x_{i1}x_{j1}$$

Looking at the indices, we can see that

$$\sum_i \sum_j x_{i1}x_{j1}=\sum_i \sum_j x_{j1}x_{i1}=[xx^T]_{ji}$$

Therefore,

$$\frac{\partial f(M)}{\partial M_{ij}}=xx^T$$