Given a probability vector $v=(v_1,\ldots,v_n)$ with $1\geq v_i\geq 0$ and $\sum_{i=1}^n v_i=1$ its entropy can be defined as: $$ H(v):=-\sum_{i=1}^nv_i\log v_i $$ I wonder what is known about transformations that leave the entropy of the vector unchanged. That is, given $v$, a map $T:\mathbb R^n \to \mathbb R^n$ that takes probability vectors to probability vectors and verifies $H(v)=H(T(v))$.

A permutation would be a trivial one. For extremal vector $(0,\ldots ,1,\ldots,0)$ this would be the only solution. However, in general, there is an infinity of valid transformations. Is anything known about the structure of these maps?

  • 2
    $\begingroup$ Not clear in general. For example there the vectors $(1/4,1/4,1/4,1/4)$ and $(1/2,1/8,1/8,1/8,1/8)$ give the same entropy, but even if you fix the size of your vector there are many that give the same entropy. I have not seen any result about such transformations $T$. $\endgroup$ – MHS Feb 13 '15 at 13:30
  • 1
    $\begingroup$ You can think of equi-entropy contours on the probability simplex. The shape of these are often shown in papers, but unfortunately, I don't have a reference on top of my head. EDIT: here's one books.google.com/… and arxiv.org/pdf/1109.6440.pdf $\endgroup$ – Memming Feb 16 '15 at 22:44
  • $\begingroup$ I understand that you can draw equientropic contours on the simplex, but I would expect (or more precisely like) points with equal entropy to have some underlying structure $\endgroup$ – Euclean Feb 17 '15 at 8:32
  • $\begingroup$ @Memming no answer so far and the bounty expires in a couple of days. If you want to expand your comment into an answer I would accept it (provide it no other answer appears) $\endgroup$ – Euclean Feb 20 '15 at 15:01

Here is a visualization of the isoentropy contours on 3 variable simplex (from Information Theoretic Learning: Renyi’s Entropy and Kernel Perspectives. Disclaimer: I contributed to a chapter in this book).

enter image description here

You could find what local changes can be applied by studying the tangent of the manifold perhaps.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.