Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to calcuate the cluster entropy to represent the distinctness of a phrase in a set of documents. I understand the meaning behind it but I don't understand the notation equation itself, as in the order it should be calculated in. Could someone please explain what it means? Here is the definition:

For given phrase w, the corresponding document set D(w) might
overlaps with other D(wi) where wi≠w. At one extreme, if D(w) is
evenly distributed in D(wi), w might be a too general phrase to be
a good salient phrase. At the other extreme, if D(w) seldom
overlaps with D(wi), w may have some distinct meaning. Take
query "jaguar" as an example, "big cats" seldom co-occur with
other salient keywords such as "car", "mac os", etc. Therefore the
corresponding documents may constitute a distinct topic.
However, "clubs" is a more general keyword which may co-occur
with both "car" and "mac os", thus it should have less salience
We use Cluster Entropy (CE) to represent the distinctness of a

$$CE = - \sum_t\frac{|D(w)\cap D(t)|}{|D(w)|} log \frac{|D(w)\cap D(t)|}{|D(w)|} $$

share|cite|improve this question

It is very simple. Here

$$CE = - \sum_t\frac{|D(w)\cap D(t)|}{|D(w)|} log \frac{|D(w)\cap D(t)|}{|D(w)|} $$

$|D(w)\cap D(t)|$ gives you the intersection of one word with all others in the text. If you divide it by the total number of the occurances of that word $D(w)$, you will get an estimated probability of intersection $p_i$.

From the definition of entropy, then you will get $$ H=\sum_i p_i log p_i $$

share|cite|improve this answer
I understand what the equation does but what I don't understand is the actual notation. Say $\frac{|D(w)\cap D(t)|}{|D(w)|}$ = 0.12; is the equation then: CE = -(0.12*(log(0.12))), thereby giving an answer of +0.9208.... – adohertyd Jul 27 '12 at 12:08
No. You will have a set of probabilities parmetrized by $t$. Per $t$ you will have $p_i$. The entropy which will finally find will be between $0$ and $log N$ where N is the cardinality of your set of probabilities. – Seyhmus Güngören Jul 27 '12 at 15:13

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.