# Maximum value of Rényi entropy

Given a discrete random variable $$X$$, which takes values in the alphabet $$\mathcal {X}$$ and is distributed according to $$p:{\mathcal {X}}\to [0,1]$$ the Shannon entropy is defined as: $$\mathrm {H} (X):=-\sum _{x\in {\mathcal {X}}}p(x)\log p(x)$$ As we know (see e.g. Prove the maximum value of entropy function) the maximum value of the Shannon entropy is $$\ln N$$ where $$N=\operatorname{card}(\mathcal X)$$.

The Rényi entropy of order $$\alpha$$, where $$\alpha \geq 0$$ and $$\alpha \neq 1$$, is defined as $$\mathrm {H} _{\alpha }(X)={\frac {1}{1-\alpha }}\log {\Bigg (}\sum _{i=1}^{N}p_{i}^{\alpha }{\Bigg )}$$ My question is: is it possible to find an upper bound also for the Rényi entropy?

The Rényi entropy for any $$\alpha \geq 0$$ is Schur concave.
This implies that the maximum is still achieved for the uniform distribution, i.e., $$H_\alpha(X) \leq \frac{1}{1-\alpha} \log \!\left(\sum_{i=1}^N \frac{1}{N^{\alpha}}\right) = \log N$$ for any $$\alpha \geq 0$$ and $$X$$ supported on a domain of size $$N$$.
Another way to find this upper bound (which one can then check is achieved for the uniform distribution in the domain) is to use the fact that $$H_\alpha(X)$$ is a non-increasing function of $$\alpha\geq 0$$, so $$H_\alpha(X) \leq H_0(X) = \log N$$.
• @Mark Schur-concavity of $f$ implies that if you have a vector of probability $x$ majorizing $y$, then $f(x)\leq f(y)$. Majorization of the uniform distribution $y$ by any other distribution $x$ is immediate from the doubly stochastic matrix characterization (see linked page on Wikipedia in the previous sentence): take this double schotastic matrix $D$ to be the (scaled by $1/N$) all-ones matrix. Jan 10, 2023 at 11:29