Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want calculate relative entropy between two strings of binary such as:

It is primarily when the lengths of two strings is same and in general when the length is not same. and how calculate $p$ and $q$. Is it related to frequency in string or need infer from some more general distribution or what?
For example what is the $q$ and $p$ here?

share|cite|improve this question
Relative entropy is defined between two probability distributions. If you only have samples, you'll need to define a probability distribution and then estimate relative entropy. Is this a homework? – Memming Nov 5 '13 at 2:46

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.