1
$\begingroup$

Kullback–Leibler divergence is written as $$\sum_{i=1}^\infty p_i \log\left(\frac{p_i}{q_i}\right).$$ With $p_i$ as a variable, now Mosek, touts that the epigraph of this divergence function, which is $\displaystyle\sum_{i=1}^\infty p_i \log\left(\frac{p_i}{q_i}\right) \leq z $, can be written as an exponential cone. Can somebody show me how?

By the way exponential cone is expressed as $ t \geq y \exp(\frac{x}{y}) $.

Some presentation slides from MOSEK gives some clue (slide/page 99), https://docs.mosek.com/slides/2017/aau/conic-opt.pdf.

Thanks for helping!!!

$\endgroup$
  • $\begingroup$ Indeed, CVX implements relative entropy using the exponential cone, but it has to rely on heuristic and not-always-reliable approximation methods to implement that. I'm looking forward to having a commercially supported solver that can handle it! $\endgroup$ – Michael Grant Mar 5 '18 at 17:21
3
$\begingroup$

Exponential cones in Mosek are not yet available. They will come in next version.

In any case, $p_i\log(p_i/q_i)\leq z_i$ is equivalent to $q_i\geq p_i \exp(-z_i/p_i)$, plus you need a constraint $\sum z_i \leq z$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.