Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

It is not difficult to show that $${n \choose \lambda n} \leq 2^{H(\lambda)n}$$ where $H$ is the binary entropy function: $$H(\alpha) = -\alpha \lg \alpha - (1-\alpha)\lg (1-\alpha)$$

I was wondering if there is an intuition about this relation between ${n \choose \lambda n}$ and the entropy of $\lambda$.

share|improve this question
add comment

1 Answer

up vote 2 down vote accepted

There is: The left-hand side is the number of states available to a system with a fixed number $\lambda n$ of particles distributed over $n$ sites whereas the right-hand side is the effective number of states available to a system of $n$ sites independently occupied with probability $\lambda$. It's intuitively clear that the latter system has more states available, since the particle number is allowed to vary. The difference is related to the one between the canonical ensemble and the grand canonical ensemble. Alternatively, you can regard the sites as possible excitations of equal energy; then the difference is related to the one between the microcanonical ensemble and the canonical ensemble.

The binary entropy $-\sum_ip_i\lg p_i$ is the binary logarithm of the effective number of states. The binary entropy of the system with fixed particle number $\lambda n$ is

$$ -\binom n{\lambda n}\left(\binom n{\lambda n}^{-1}\lg\binom n{\lambda n}^{-1}\right)=\lg\binom n{\lambda n}, $$

whereas the binary entropy of the system with given proportion $\lambda$ is the sum of the binary entropies of the $n$ independent systems, and $H(\lambda)$ is precisely the entropy of a site with two states, occupied with probability $\lambda$ and unoccupied with probability $1-\lambda$; the sum for all $n$ sites is $H(\lambda)n$.

share|improve this answer
    
Very nice, thank you :) –  Kaveh Oct 24 '12 at 14:19
    
@Kaveh: You're welcome! –  joriki Oct 24 '12 at 14:51
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.