0
$\begingroup$

I want to prove that $log(\sum_{i=1}^{n}{e^{x_i}})$ is convex. To do that I would need to find its gradient and Hessian, but I'm not sure how to take derivatives of summations with two variables.

EDIT: I was mistaken and there is only one variable $x$. I believe the first derivative would thus be, $\frac{d}{dx}$ $log(\sum_{i=1}^{n}{e^{x_i}})$ = $\frac{1}{\sum_{i=1}^{n}{e^{x_i}}}$. Is this correct? Also, from there, how would I take the derivative of a summation of one variable to get the second derivative?

$\endgroup$
  • 1
    $\begingroup$ Ther is only summation with one variable, isn't there? $\frac{\mathrm d}{\mathrm dx}\sum_{i=1}^nf_i(x)=\sum_{i=1}^n\frac{\mathrm d}{\mathrm dx}f_i(x)$ $\endgroup$ – Hagen von Eitzen Sep 4 '17 at 18:33
  • $\begingroup$ Oh I see. I was confused by the $n$. I'll try and see what I get now. $\endgroup$ – dirtysocks45 Sep 4 '17 at 18:36
  • 2
    $\begingroup$ for derivative of log you need to derivate the sum too $ \ \dot{ log(f(x))} = \frac {f'(x)} { f(x) } $ $\endgroup$ – Isham Sep 4 '17 at 18:44
  • 1
    $\begingroup$ @dirtysocks45, it looks like you have some multivariate calculus prerequisites to master. If you happen to be looking at the book Convex Optimization by Boyd & Vandenberghe, check out Appendix A, "Mathematical Background". Most of that material should be a review. If it's not, you might get by just by learning the material there, but it will be tougher. $\endgroup$ – Michael Grant Sep 4 '17 at 23:47
  • 1
    $\begingroup$ I'm afraid I am unfamiliar with the text. $\endgroup$ – Michael Grant Sep 5 '17 at 21:10
1
$\begingroup$

Note that $$\frac{\partial}{\partial x_k} \log \left(\sum_{i=1}^{n} e^{x_i}\right)=\frac{1}{\sum_{i=1}^{n} e^{x_i}}\cdot \frac{\partial}{\partial x_k}\left(\sum_{i=1}^{n} e^{x_i}\right) = \frac{e^{x_k}}{\sum_{i=1}^{n} e^{x_i}}.$$ Consequently, $$\frac{\partial^2}{\partial x_k^2}\log \left(\sum_{i=1}^{n} e^{x_i}\right)=\frac{e^{x_k}\left(\sum_{i=1}^{n} e^{x_i}-e^{x_k}\right)}{\left(\sum_{i=1}^{n} e^{x_i}\right)^2},$$ and $$\frac{\partial^2}{\partial x_k \partial x_j}\log \left(\sum_{i=1}^{n} e^{x_i}\right)=-\frac{e^{x_k}e^{x_j}}{\left(\sum_{i=1}^{n} e^{x_i}\right)^2},$$ where $j \neq k$, and $1 \le j,k \le n$.

$\endgroup$
  • $\begingroup$ Is there a way to take the derivative of the whole function instead of partial derivatives over the entire summation? $\endgroup$ – dirtysocks45 Sep 4 '17 at 19:18
  • $\begingroup$ I get that now, but what about taking the derivative of the whole function? $\endgroup$ – dirtysocks45 Sep 4 '17 at 19:19
  • $\begingroup$ What do you mean by the derivative of the whole function? I believe you meant the gradient. If that is the case, then it is just a vector of the partial derivatives. $\endgroup$ – Math Lover Sep 4 '17 at 19:21
  • 1
    $\begingroup$ Just to be clear, $\sum_{i=1}^{n} e^{x_i}$ is a function of $n$ variables? $\endgroup$ – Math Lover Sep 4 '17 at 19:37
  • 1
    $\begingroup$ So it is a function of $n$ variables, $x_1, x_2, \cdots, x_n$. That's why you are asked to compute the gradient vector and the Hessian matrix. Also, $\frac{\partial}{\partial x_i}[ f(x_i) + g(x_j)] = \frac{\partial}{\partial x_i} f(x_i)$. $\endgroup$ – Math Lover Sep 4 '17 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.