Economist here. In my current research I often encounter the concepts of entropy and means, and I find some connections interesting (for my purposes).

The context is a set of $n$ nonnegative numbers $\{a_1,\dots,a_n\}$, which can be normalized by the sum of the $a_i$'s to get $\{x_1,\dots,x_n\}$. These $x_i$'s serve as weights in the context of means, and they serve as probabilities out of which entropy can be calculated. I find relationships between entropy and means, particularly between

\begin{align*} \mathcal{A}\left(a_{i},\frac{1}{n}\right) & =\frac{1}{n}\sum a_{i} & \underset{\textrm{(i.e. equal weights }\frac{1}{n})}{\textrm{unweighted arithmetic mean}}\\ \mathcal{G}\left(a_{i},x_{i}\right) & =\prod a_{i}^{x_{i}} & \underset{\textrm{(with weights }x_{i}\textrm{ summing to one)}}{\textrm{weighted geometric mean}}\\ H(x_{i}) & =-\sum_{i=1}^{n}x_{i}\ln x_{i} & \underset{\textrm{(same }x_{i}\textrm{ as above, i.e. probabilities)}}{\textrm{Shannon entropy}} \end{align*}


  1. In general, how are entropy and means related? I can't think I'm the first one to notice.
  2. The only thing I can think of is the inequality $\mathcal{A}\ge\mathcal{G}$ between the arithmetic and geometric means; if I am right, the difference or divergence between these two means relates to the Shannon entropy. But not so fast: the inequality of means is valid when both the arithmetic and geometric means have the same weights (either $\frac{1}{n}$ or $x_i$)--which is not the case here. Any thoughts greatly appreciated.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.