2
$\begingroup$

Economist here. In my current research I often encounter the concepts of entropy and means, and I find some connections interesting (for my purposes).

The context is a set of $n$ nonnegative numbers $\{a_1,\dots,a_n\}$, which can be normalized by the sum of the $a_i$'s to get $\{x_1,\dots,x_n\}$. These $x_i$'s serve as weights in the context of means, and they serve as probabilities out of which entropy can be calculated. I find relationships between entropy and means, particularly between

\begin{align*} \mathcal{A}\left(a_{i},\frac{1}{n}\right) & =\frac{1}{n}\sum a_{i} & \underset{\textrm{(i.e. equal weights }\frac{1}{n})}{\textrm{unweighted arithmetic mean}}\\ \mathcal{G}\left(a_{i},x_{i}\right) & =\prod a_{i}^{x_{i}} & \underset{\textrm{(with weights }x_{i}\textrm{ summing to one)}}{\textrm{weighted geometric mean}}\\ H(x_{i}) & =-\sum_{i=1}^{n}x_{i}\ln x_{i} & \underset{\textrm{(same }x_{i}\textrm{ as above, i.e. probabilities)}}{\textrm{Shannon entropy}} \end{align*}

Questions:

  1. In general, how are entropy and means related? I can't think I'm the first one to notice.
  2. The only thing I can think of is the inequality $\mathcal{A}\ge\mathcal{G}$ between the arithmetic and geometric means; if I am right, the difference or divergence between these two means relates to the Shannon entropy. But not so fast: the inequality of means is valid when both the arithmetic and geometric means have the same weights (either $\frac{1}{n}$ or $x_i$)--which is not the case here. Any thoughts greatly appreciated.
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.