12
$\begingroup$

Suppose we have a linear scalar field $f : \mathbb{R}^{n \times m} \to \mathbb{R}$ defined by

$$ f(M) := x^T M x $$

where $x \in \mathbb{R}^n$. What is the gradient of $f(M)$ with respect to $M$? I think it is $xx^T$, but why?

$\endgroup$
3
  • 2
    $\begingroup$ Since $f$ is linear, its differential is equal to $f$... $\endgroup$
    – Gibbs
    Nov 1, 2018 at 0:35
  • 1
    $\begingroup$ Can you write down the calculation for me? $\endgroup$
    – user494522
    Nov 1, 2018 at 0:38
  • $\begingroup$ Related $\endgroup$ Apr 26, 2021 at 1:48

2 Answers 2

6
$\begingroup$

$$ f(M)=\sum_{i,j}x_iM_{i,j}x_j. $$ Therefore $$ \frac{\partial f(M)}{\partial M_{i,j}} = x_ix_j = (xx^T)_{i,j}. $$

$\endgroup$
2
  • 1
    $\begingroup$ Could you explain the last step? $\endgroup$
    – user494522
    Nov 1, 2018 at 0:40
  • $\begingroup$ What is $xx^T$ for you? $\endgroup$
    – Federico
    Nov 1, 2018 at 0:41
5
$\begingroup$

The strategy is to write the expression as a scalar using index notation, take the derivative, and re-write in matrix form.

Note that to write the function as a summation of matrices we have to write just one scalar as a matrix multiplication because the function is scalar: $$ f(M)= [x^TMx]_{11}= \sum_i x_{i1}[Mx]_{i1} $$ At first summation, we have $x_{i1}$ because we have $x^T$ as the first term.

Now expand $[Mx]_{i1}$ $$ f(M)= [x^TMx]_{11}= \sum_i x_{i1}\sum_j M_{ij}x_{j1}=\sum_i \sum_j x_{i1} M_{ij}x_{j1} $$

Now take the derivative with respect to $M_{ij}$

$$ \frac{\partial f(M)}{\partial M_{ij}}=\sum_i \sum_j x_{i1}x_{j1} $$

Looking at the indices, we can see that

$$ \sum_i \sum_j x_{i1}x_{j1}=\sum_i \sum_j x_{j1}x_{i1}=[xx^T]_{ji} $$

Therefore,

$$ \frac{\partial f(M)}{\partial M_{ij}}=xx^T $$

$\endgroup$

You must log in to answer this question.