11
$\begingroup$

Suppose we have the following quadratic form

$$ f(M)=x^TMx $$ where $f: \mathbb{R}^{n \times m} \rightarrow \mathbb{R}$, and $x \in \mathbb{R}^n$. As it is obvious, this function is linear in $M$. What is the derivative of $f(M)$ with respect to $M$. I think it is $xx^T$ but why?

$\endgroup$
3
  • 2
    $\begingroup$ Since $f$ is linear, its differential is equal to $f$... $\endgroup$
    – Gibbs
    Nov 1, 2018 at 0:35
  • 1
    $\begingroup$ Can you write down the calculation for me? $\endgroup$
    – Saeed
    Nov 1, 2018 at 0:38
  • $\begingroup$ Related $\endgroup$ Apr 26, 2021 at 1:48

2 Answers 2

7
$\begingroup$

$$ f(M)=\sum_{i,j}x_iM_{i,j}x_j. $$ Therefore $$ \frac{\partial f(M)}{\partial M_{i,j}} = x_ix_j = (xx^T)_{i,j}. $$

$\endgroup$
2
  • $\begingroup$ Could you explain the last step? $\endgroup$
    – Saeed
    Nov 1, 2018 at 0:40
  • $\begingroup$ What is $xx^T$ for you? $\endgroup$
    – Federico
    Nov 1, 2018 at 0:41
5
$\begingroup$

The strategy is to write the expression as a scalar using index notation, take the derivative, and re-write in matrix form.

Note that to write the function as a summation of matrices we have to write just one scalar as a matrix multiplication because the function is scalar: $$ f(M)= [x^TMx]_{11}= \sum_i x_{i1}[Mx]_{i1} $$ At first summation, we have $x_{i1}$ because we have $x^T$ as the first term.

Now expand $[Mx]_{i1}$ $$ f(M)= [x^TMx]_{11}= \sum_i x_{i1}\sum_j M_{ij}x_{j1}=\sum_i \sum_j x_{i1} M_{ij}x_{j1} $$

Now take the derivative with respect to $M_{ij}$

$$ \frac{\partial f(M)}{\partial M_{ij}}=\sum_i \sum_j x_{i1}x_{j1} $$

Looking at the indices, we can see that

$$ \sum_i \sum_j x_{i1}x_{j1}=\sum_i \sum_j x_{j1}x_{i1}=[xx^T]_{ji} $$

Therefore,

$$ \frac{\partial f(M)}{\partial M_{ij}}=xx^T $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.