# Convexity of the exponential of the negative Renyi entropy

For $$r\ge -1$$, the exponential of the negative Renyi entropy is defined as $$M(p):=\Big(\sum_i p_i^{1+r}\Big)^{\frac1r},$$ for a probability measure as tuples $$p:=(p_i)_i$$ I would like to prove the convexity of $$M(\cdot)$$, or $$M(ap+bq)\le aM(p)+bM(q),$$ $$\forall\,a+b=1 \wedge a,b\ge0$$, and two probability measures $$p$$ and $$q$$ with the same cardinalities.

For $$r>0$$, I can show the convexity via the Minkowski inequality for $$\big(\sum_i x_i^{1+r}\big)^{\frac1{1+r}}$$ then the convexity of $$f(x):=x^{1+\frac1r}$$.

But how would one show the convexity for $$-1? The above technique does not work since the inequality signs from the two steps point in the opposite directions.

• While you have proven it already there is an even easier way. Do note that $h(x) = x^{1/r}$ is convex and nonincreasing for $-1 < r < 0$ on $\mathbb{R}_+$, and $g(x) = \sum_{i=1}^n x_i^{1+r}$ is concave on $\mathbb{R}^n_+$ (very simple 2nd derivative tests). Therefore, $f = h\circ g$ must be convex, see page 84 in Vandenberghe and Boyd's "Convex Optimization". Jan 25 at 0:12
• @V.S.e.H.: Right and thank you. But... darn! I was seeking a composition just like that. I should have thought about it. Why don't you write it up as an answer for posterity and get some credit? :-)
– Hans
Jan 25 at 8:35
• Yessir, as you wish :) Jan 25 at 9:31

This is actually easier than I thought. I came to the same idea slightly before @IosifPinelis suggested it.

Consider $$x>0$$, $$h\in \mathbf R^n$$ and $$S(p+th):=M(p+th)^r=\sum_i(p_i+th_i)^{1+r}.$$ $$\frac d{dt}S(p+th)^{\frac1r}=\Big(1+\frac1r\Big)S^{\frac1r-1}\sum_i(p_i+th_i)^rh_i,$$ \begin{align} \frac{d^2}{dt^2}M(p+th)&=\frac{d^2}{dt^2}S(p+th)^{\frac1r} \\ &=\Big(1+\frac1r\Big)S^{\frac1r-2}\Big(\frac{1-r^2}{r}\big(\sum_i(p_i+th_i)^rh_i\big)^2+rS\sum_i(p_i+th_i)^{r-1}h_i^2\Big) \\ &=(1+r)S^{\frac1r-2}\Big(\sum_ix_i^{r+1}\sum_ix_i^{r-1}h_i^2-\frac{r^2-1}{r^2}\big(\sum_ix_i^rh_i\big)^2\Big), \end{align} where $$x_i:=p_i+th_i$$. Now set $$t=0$$.

1. For $$r\in[-1,1]$$, the last expression is obviously positive.

2. For $$r>1$$, aside from the method I stated in my question, we can prove it via the Cauchy-Schwarz inequality as follows. $$\Big(\sum_ix_i^{\frac {r+1}2}\big(x_i^{\frac{r-1}2}h_i\big)\Big)^2\le \sum_ix_i^{r+1}\sum_ix_i^{r-1}h_i^2.$$ Together with $$\frac{r^2-1}{r^2}<1$$, $$\frac{d^2}{dt^2}M(p+th)\big|_{t=0}>0.$$

• (+1) Congratulations. Jan 25 at 0:46

Let $$h(t) = t^{1/r}$$ and $$g(x) = \sum_{i=1}^n x_i^{1+r}$$, $$0< r < -1$$.

We have that $$h'(t) = \frac{t^{1/r-1}}{r} \leq 0$$ for all $$0 \leq t$$, so $$h$$ is nonincreasing on $$\mathbb{R}_+$$, and $$h''(t) = (1/r-1)\frac{t^{1/r-2}}{r} \geq 0$$ for all $$0\leq t$$, so $$h$$ is convex.

Similarly, $$u(t) = t^{1+r}$$ is concave on $$0 \leq t$$, since $$u''(t) = r(1+r)t^{r-1} \leq 0$$, which implies that $$g(x)$$ is concave (sum of component-wise concave functions).

Now, $$g(\theta x + (1-\theta)y)\geq \theta g(x) + (1-\theta)g(y),$$ so $$h(g(\theta x + (1-\theta)y)) \leq h(\theta g(x) + (1-\theta)g(y)) \leq \theta h(g(x)) + (1-\theta)h(g(y))$$.

Therefore $$f=h\circ g$$ is convex, as desired.