Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm computing cosine similarities between 2 vectors.

These vectors are information retrieval query and document representations respectively.

They have been computed using tf/idf weights.

Since my documents have different length, tf/idf weights are theoretically unbounded.

The question is: is cosine similarity still a valid measure ? Can I compare several cosine similarities for each doc ?


share|cite|improve this question
For the uninformed like me: – t.b. Mar 29 '11 at 9:29
Try asking at It's the q&a forum for machine learning related topics, including information retrieval. And it's just a hunch but if your vectors are defined over the entire vocabulary, and elements corresponding to words that don't appear in the document are given a value of zero, then I don't see why you'd have trouble doing cosine. – JasonMond May 28 '11 at 18:45
up vote 1 down vote accepted

If I read Wikipedia right, tf/idf is not unbounded. tf $\le 1$ (would be 1 only if the document had all words the same) and idf $\le \log N, N$ the number of documents, with equality if only one document has the term. Despite the slash in tf/idf, these are multiplied so the limit is $\log N$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.