Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Consider $n$ independent and identically distributed random variables $ \{X_i\}_{i=1,...n} $ with support on some interval $[a,b]$ and its $n$'th order statistic $\max_{i \in \{1,...n\}} X_i$ . The entropy of the maximum is

$$ - \int_a^b F^n(x) \ln F^n(x) dx ,$$ where $F(x)= \Pr (X \le x) $. It seems natural that the entropy should be decreasing in $n$ (just think about $n$ very large). Is this a known result?

I did in fact prove that the entropy is monotone, but the proof turned out to be lengthy and messy. I would expect that there is a simple argument. Does anyone know?

share|improve this question
1  
The integral quoted does not correspond to the Shannon entropy of the maximal order statistics, I am afraid. By definition the entropy $S_Z = - \int \ln(f_Z(z)) f_Z(z) \mathrm{d} z$, and for the $\max$, $f_{X_{n:n}}(x) = \left(F_X(x)^n\right)^\prime$. –  Sasha Jul 15 '12 at 19:36
1  
If X has a density as Sasha said the cumulative distribution for the maximum is F$^n$(x) and the density f(x) =nF$^n$$^-$$^1(x)$F'(x). –  Michael Chernick Jul 15 '12 at 19:45
1  
Now ln f(x)= ln n + ln F'(x) + (n-1)ln F(x). –  Michael Chernick Jul 15 '12 at 19:50
    
Sorry, I did use the wrong definition. -S –  Stephan Jul 15 '12 at 20:03
add comment

1 Answer 1

up vote 1 down vote accepted

No, the entropy is not monotone. For example, consider $F_X(x) = x^{1/N}$ on $[0,1]$. Then $\max(X_1,\ldots,X_N)$ is uniform on $[0,1]$. The entropy of $\max(X_1,\ldots,X_n)$ increases as a function of $n$ for $1 \le n \le N$, reaching $0$ at $n=N$, then decreases after that.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.