Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider $n$ independent and identically distributed random variables $ \{X_i\}_{i=1,...n} $ with support on some interval $[a,b]$ and its $n$'th order statistic $\max_{i \in \{1,...n\}} X_i$ . The entropy of the maximum is

$$ - \int_a^b F^n(x) \ln F^n(x) dx ,$$ where $F(x)= \Pr (X \le x) $. It seems natural that the entropy should be decreasing in $n$ (just think about $n$ very large). Is this a known result?

I did in fact prove that the entropy is monotone, but the proof turned out to be lengthy and messy. I would expect that there is a simple argument. Does anyone know?

share|cite|improve this question
The integral quoted does not correspond to the Shannon entropy of the maximal order statistics, I am afraid. By definition the entropy $S_Z = - \int \ln(f_Z(z)) f_Z(z) \mathrm{d} z$, and for the $\max$, $f_{X_{n:n}}(x) = \left(F_X(x)^n\right)^\prime$. – Sasha Jul 15 '12 at 19:36
If X has a density as Sasha said the cumulative distribution for the maximum is F$^n$(x) and the density f(x) =nF$^n$$^-$$^1(x)$F'(x). – Michael Chernick Jul 15 '12 at 19:45
Now ln f(x)= ln n + ln F'(x) + (n-1)ln F(x). – Michael Chernick Jul 15 '12 at 19:50
Sorry, I did use the wrong definition. -S – Nahpetz Jul 15 '12 at 20:03
up vote 1 down vote accepted

No, the entropy is not monotone. For example, consider $F_X(x) = x^{1/N}$ on $[0,1]$. Then $\max(X_1,\ldots,X_N)$ is uniform on $[0,1]$. The entropy of $\max(X_1,\ldots,X_n)$ increases as a function of $n$ for $1 \le n \le N$, reaching $0$ at $n=N$, then decreases after that.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.