# Quantifying how "concentrated" a distribution is

Consider $$n\in\mathbb{N}$$ buckets with infinite capacity and let $$x\in[0,1]^n$$ with $$\sum_{i=1}^n x_i=1$$ be a way to distribute 1 litre of liquid across all $$n$$ buckets, i.e. $$x_i$$ is the amount of litres in the $$i$$-th bucket. Denote $$S:= \{x\in[0,1]^n\vert \sum_{i=1}^n x_i = 1\}$$. I would like to quantifiy arithmetically "how much the liquid is concentrated" by using some arithmetic function $$f:S \to [0,1]$$. Here are the two main requirements:

1. $$f(x)=0\$$ if and only if $$\ x_i=\frac{1}{n}\$$ for all $$\ 1\leq i\leq n$$.
2. $$f(x)=1\$$ if and only if $$\ x_i=1\$$ for some $$\ 1\leq i\leq n$$.

I am aware of how open this question is, but I am looking specifically for an "arithmetically pretty" solution.

My idea was considering the sum of squares $$g(x)=\sum_{i=1}^n x_i^2 \in [\frac{1}{n}, 1]$$ and scaling it appropriately by choosing: $$f(x)=\frac{g(x)-\frac{1}{n}}{1 - \frac{1}{n}}=\frac{n\cdot g(x)-1}{n - 1}\in[0,1]$$ I am not a huge fan of how the end result looks.

• This reminds me of the different ways to measure sparsity. This paper lists a number of them.
– p.s.
Commented Jun 22 at 0:47

These conditions are satisfied by a suitably rescaled version of the entropy

$$H(x) = \sum_i - x_i \log x_i.$$

Loosely speaking the entropy is a measure of how "random" or "uncertain" (or "dispersed," the opposite of concentrated) the $$x_i$$ are when interpreted as a probability distribution. The entropy is famously maximized at the uniform distribution $$x_i = \frac{1}{n}$$, where it takes the value $$\log n$$, and minimized at the point distributions $$x_i = 1$$, where it takes the value $$0$$. This means we can take

$$\boxed{ f(x) = 1 - \frac{H(x)}{\log n} }.$$

(This can be made to look a little nicer by absorbing the $$n$$ into the base of the logarithm; that is, we can use $$\sum x_i \log_n x_i$$. But I don't think this matters too much either way. Actually it is important and meaningful that the entropy takes the value $$\log n$$ for the uniform distribution and is not just rescaled to take the value $$1$$.)

The entropy has various pleasant properties which make precise some senses in which it is a "good measure of uncertainty," some of which uniquely characterize it, which you can check out on Wikipedia. There are also generalizations like the Rényi entropy. This is related to the study of diversity indices and you can see, for example, Leinster's Entropy and Diversity: An Axiomatic Approach for a discussion of this.

$$f(x) = \frac{n}{n-1}\sum_{i=1}^n \left(x_i - \frac{1}{n} \right)^2$$
The summation measures the deviance from a uniform allocation of $$x_i = \frac{1}{n}$$, and the factor $$\frac{n}{n-1}$$ scales the result so that the maximum value is 1.