# Kullback–Leibler divergence as Exponential cone

Kullback–Leibler divergence is written as $$\sum_{i=1}^\infty p_i \log\left(\frac{p_i}{q_i}\right).$$ With $p_i$ as a variable, now Mosek, touts that the epigraph of this divergence function, which is $\displaystyle\sum_{i=1}^\infty p_i \log\left(\frac{p_i}{q_i}\right) \leq z$, can be written as an exponential cone. Can somebody show me how?

By the way exponential cone is expressed as $t \geq y \exp(\frac{x}{y})$.

Some presentation slides from MOSEK gives some clue (slide/page 99), https://docs.mosek.com/slides/2017/aau/conic-opt.pdf.

Thanks for helping!!!

• Indeed, CVX implements relative entropy using the exponential cone, but it has to rely on heuristic and not-always-reliable approximation methods to implement that. I'm looking forward to having a commercially supported solver that can handle it! – Michael Grant Mar 5 '18 at 17:21

## 1 Answer

Exponential cones in Mosek are not yet available. They will come in next version.

In any case, $p_i\log(p_i/q_i)\leq z_i$ is equivalent to $q_i\geq p_i \exp(-z_i/p_i)$, plus you need a constraint $\sum z_i \leq z$.