3
$\begingroup$

For $r\ge -1$, the exponential of the negative Renyi entropy is defined as $$M(p):=\Big(\sum_i p_i^{1+r}\Big)^{\frac1r},$$ for a probability measure as tuples $p:=(p_i)_i$ I would like to prove the convexity of $M(\cdot)$, or $$M(ap+bq)\le aM(p)+bM(q),$$ $\forall\,a+b=1 \wedge a,b\ge0$, and two probability measures $p$ and $q$ with the same cardinalities.

For $r>0$, I can show the convexity via the Minkowski inequality for $\big(\sum_i x_i^{1+r}\big)^{\frac1{1+r}}$ then the convexity of $f(x):=x^{1+\frac1r}$.

But how would one show the convexity for $-1<r<0$? The above technique does not work since the inequality signs from the two steps point in the opposite directions.

$\endgroup$
3
  • 1
    $\begingroup$ While you have proven it already there is an even easier way. Do note that $h(x) = x^{1/r}$ is convex and nonincreasing for $-1 < r < 0$ on $\mathbb{R}_+$, and $g(x) = \sum_{i=1}^n x_i^{1+r}$ is concave on $\mathbb{R}^n_+$ (very simple 2nd derivative tests). Therefore, $f = h\circ g$ must be convex, see page 84 in Vandenberghe and Boyd's "Convex Optimization". $\endgroup$
    – V.S.e.H.
    Jan 25 at 0:12
  • $\begingroup$ @V.S.e.H.: Right and thank you. But... darn! I was seeking a composition just like that. I should have thought about it. Why don't you write it up as an answer for posterity and get some credit? :-) $\endgroup$
    – Hans
    Jan 25 at 8:35
  • $\begingroup$ Yessir, as you wish :) $\endgroup$
    – V.S.e.H.
    Jan 25 at 9:31

2 Answers 2

2
$\begingroup$

This is actually easier than I thought. I came to the same idea slightly before @IosifPinelis suggested it.


Consider $x>0$, $h\in \mathbf R^n$ and $$S(p+th):=M(p+th)^r=\sum_i(p_i+th_i)^{1+r}.$$ $$\frac d{dt}S(p+th)^{\frac1r}=\Big(1+\frac1r\Big)S^{\frac1r-1}\sum_i(p_i+th_i)^rh_i,$$ \begin{align} \frac{d^2}{dt^2}M(p+th)&=\frac{d^2}{dt^2}S(p+th)^{\frac1r} \\ &=\Big(1+\frac1r\Big)S^{\frac1r-2}\Big(\frac{1-r^2}{r}\big(\sum_i(p_i+th_i)^rh_i\big)^2+rS\sum_i(p_i+th_i)^{r-1}h_i^2\Big) \\ &=(1+r)S^{\frac1r-2}\Big(\sum_ix_i^{r+1}\sum_ix_i^{r-1}h_i^2-\frac{r^2-1}{r^2}\big(\sum_ix_i^rh_i\big)^2\Big), \end{align} where $x_i:=p_i+th_i$. Now set $t=0$.

  1. For $r\in[-1,1]$, the last expression is obviously positive.

  2. For $r>1$, aside from the method I stated in my question, we can prove it via the Cauchy-Schwarz inequality as follows. $$\Big(\sum_ix_i^{\frac {r+1}2}\big(x_i^{\frac{r-1}2}h_i\big)\Big)^2\le \sum_ix_i^{r+1}\sum_ix_i^{r-1}h_i^2.$$ Together with $\frac{r^2-1}{r^2}<1$, $$\frac{d^2}{dt^2}M(p+th)\big|_{t=0}>0.$$

$\endgroup$
1
  • $\begingroup$ (+1) Congratulations. $\endgroup$
    – River Li
    Jan 25 at 0:46
0
$\begingroup$

Let $h(t) = t^{1/r}$ and $g(x) = \sum_{i=1}^n x_i^{1+r}$, $0< r < -1$.

We have that $h'(t) = \frac{t^{1/r-1}}{r} \leq 0$ for all $0 \leq t$, so $h$ is nonincreasing on $\mathbb{R}_+$, and $h''(t) = (1/r-1)\frac{t^{1/r-2}}{r} \geq 0$ for all $0\leq t$, so $h$ is convex.

Similarly, $u(t) = t^{1+r}$ is concave on $0 \leq t$, since $u''(t) = r(1+r)t^{r-1} \leq 0$, which implies that $g(x)$ is concave (sum of component-wise concave functions).

Now, $g(\theta x + (1-\theta)y)\geq \theta g(x) + (1-\theta)g(y),$ so $h(g(\theta x + (1-\theta)y)) \leq h(\theta g(x) + (1-\theta)g(y)) \leq \theta h(g(x)) + (1-\theta)h(g(y))$.

Therefore $f=h\circ g$ is convex, as desired.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.