I would like to clarify the Gradient and Hessian of $$g(x) = \sum_i g_i(x_i) ,$$ where $x = [x_1,\ldots,x_N]^T \in \mathbb{R}^N$.

  • Is the gradient of $g(x)$ correct? $$\nabla g(x) = \left[ \nabla g_1(x_1), \ldots, \nabla g_i(x_i), \ldots, \nabla g_N(x_N) \right]^T \ .$$
  • Is the Hessian of $g(x)$ correct? $$\mathcal{H}\left( g(x) \right) = {\rm Diag} \left\{ \nabla^2 g_1(x_1), \ldots, \nabla^2 g_i(x_i), \ldots, \nabla^2 g_N(x_N) \right\} \ ,$$ where ${\rm Diag}\{ \cdot \}$ creates the diagonal matrix of the elements.
  • 1
    $\begingroup$ The gradient of $g$ is an $n$ dimensional vector whereas here you defined the gradient of $g$ as a sum of $1$ dimensional vectors, so the gradient is : $(\frac{\partial g_1}{\partial x_1}, ..., \frac{\partial g_n}{\partial x_n})$ $\endgroup$ – Thinking Jan 28 at 18:30
  • $\begingroup$ Oh Thank you. How about the Hessian? $\endgroup$ – learning Jan 28 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.