Hessian of log-sum-exp $f(z) = \operatorname{log} \sum_{i=1}^n z_i$, find $\nabla^2f(z)$

Question

Let $f(z) = \operatorname{log} \sum_{i=1}^n z_i = \operatorname{log} 1^Tz$.

This problem comes from the following famous theorem:

---

My work:
The following step is just consider one entry of the Hessian matrix:

$$\nabla^2 f(z)_{ij} = \frac{\partial f(z)}{\partial z_i\partial z_j} = \frac{\partial}{\partial z_i}(\frac{\partial f(z)}{\partial z_j}) = \frac{\partial}{\partial z_i}(\frac{1}{1^Tz})=-\frac{1}{(1^Tz)^2}$$
I have no idea how to obtain the desired form. How to obtain the other term such as $\operatorname{diag(z)}$. Moreover, the numerator here is $1$.

Answer 1

Note that you have to take the derivative with respect to $x$ if you want to check convexity in $x$. This gives some more terms form the chain rule: We have $$ \frac{\partial}{\partial x_j} f(x) = \frac{\partial}{\partial z_j} \log 1^tz \cdot \frac{\partial z_j}{\partial x_j} = \frac 1{1^tz} z_j $$ Hence, \begin{align*} \frac{\partial^2 f}{\partial x_i \partial x_j} &= \frac{\partial}{\partial z_i} \left(\frac{z_j}{1^tz}\right) \cdot \frac{\partial z_i}{\partial x_i}\\ &=\frac{\delta_{ij}1^tz - z_j}{(1^t z)^2} \cdot \exp x_i\\ &= \frac{\delta_{ij}z_i\cdot 1^tz - z_iz_j}{(1^t z)^2}\\ &= \frac{\delta_{ij}z_i}{1^tz} - \frac{z_iz_j}{(1^tz)^2}\\ &= \left(\frac 1{1^tz} \mathop{\rm diag}(z) - \frac{1}{(1^tz)^2} zz^t\right)_{i,j} \end{align*}