1
$\begingroup$

Let $f(x) = \sum_{1:n} c_ix_i \cdot \sum_{1:n} d_ix_i$.

How do I take the gradient of this function?

Tried the product rule

$\nabla f = (\nabla\sum_{1:n} c_ix_i)\sum_{1:n}d_ix_i + (\nabla\sum_{1:n}d_ix_i)\sum_{1:n} c_ix_i \\ \Rightarrow \nabla_i f = c_i\sum d_jx_j + d_i\sum c_jx_j$

But I'm not sure I got it right.

$\endgroup$
  • $\begingroup$ I assume you know the definition of gradient. Where are you stuck? Note that $$\sum_{i=1}^n c_ix_i\sum_{j=1}^n d_jx_j=2\sum_{1\leqslant i<j\leqslant n}c_id_jx_ix_j+\sum_{i=1}^n c_id_i x_i^2$$ $\endgroup$ – Pedro Tamaroff Jan 12 '15 at 10:24
  • $\begingroup$ Maybe the product rule? $(u\cdot v)'=u\cdot v'+v\cdot u'$ ? $\endgroup$ – Pixel Jan 12 '15 at 10:25
  • $\begingroup$ That was news to me. I tried the product rule for gradients, but not sure if I got it right. Will edit $\endgroup$ – Benjamin Lindqvist Jan 12 '15 at 10:25
4
$\begingroup$

Hint: $$f(x) = c^T x \cdot d^T x = x^T cd^T x$$ The gradient of $x^T A x$ is given by $(A+A^T) x$ so $$\nabla f(x) = (cd^T + dc^T) x$$ Verify this by explicitly writing $$f(x) = \left(\sum_{i=1}^n c_i x_i \right) \cdot \left( \sum_{j=1}^n d_j x_j \right) = \sum_{i,j=1}^n x_i c_id_j x_j$$ And using linearity of the derivative plus the chain rule, remembering that $\frac{\partial x_i}{\partial x_j} = \delta_{ij}$ the Kronecker-Delta. Notice that I have used different index names for the two sums so that I can write the product as a double sum of products.

$\endgroup$
  • $\begingroup$ Very nice... Those matrix differential identities were poorly covered in the course I took. $\endgroup$ – Benjamin Lindqvist Jan 12 '15 at 10:35
  • $\begingroup$ @luegofuego Happy to help. Did anything remain unclear? If so, feel free to ask for further clarification. If not, please mark the question as answered so others know that you don't need any further help :) $\endgroup$ – AlexR Jan 12 '15 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.