13
$\begingroup$

Let $f(z) = \operatorname{log} \sum_{i=1}^n z_i = \operatorname{log} 1^Tz$.

This problem comes from the following famous theorem:

enter image description here


My work:
The following step is just consider one entry of the Hessian matrix:

$$\nabla^2 f(z)_{ij} = \frac{\partial f(z)}{\partial z_i\partial z_j} = \frac{\partial}{\partial z_i}(\frac{\partial f(z)}{\partial z_j}) = \frac{\partial}{\partial z_i}(\frac{1}{1^Tz})=-\frac{1}{(1^Tz)^2}$$
I have no idea how to obtain the desired form. How to obtain the other term such as $\operatorname{diag(z)}$. Moreover, the numerator here is $1$.

$\endgroup$
10
$\begingroup$

Note that you have to take the derivative with respect to $x$ if you want to check convexity in $x$. This gives some more terms form the chain rule: We have $$ \frac{\partial}{\partial x_j} f(x) = \frac{\partial}{\partial z_j} \log 1^tz \cdot \frac{\partial z_j}{\partial x_j} = \frac 1{1^tz} z_j $$ Hence, \begin{align*} \frac{\partial^2 f}{\partial x_i \partial x_j} &= \frac{\partial}{\partial z_i} \left(\frac{z_j}{1^tz}\right) \cdot \frac{\partial z_i}{\partial x_i}\\ &=\frac{\delta_{ij}1^tz - z_j}{(1^t z)^2} \cdot \exp x_i\\ &= \frac{\delta_{ij}z_i\cdot 1^tz - z_iz_j}{(1^t z)^2}\\ &= \frac{\delta_{ij}z_i}{1^tz} - \frac{z_iz_j}{(1^tz)^2}\\ &= \left(\frac 1{1^tz} \mathop{\rm diag}(z) - \frac{1}{(1^tz)^2} zz^t\right)_{i,j} \end{align*}

$\endgroup$
2
  • 1
    $\begingroup$ Can you explain a bit about $\delta_{ij}$? $\endgroup$ – user436661 Feb 6 '20 at 8:38
  • $\begingroup$ The so called Kronecker symbol, a shorthand for $\delta_{ij} = 1$ iff $i=j$, otherwise $\delta_{ij} = 0$. $\endgroup$ – martini Jun 12 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.