Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I want calculate relative entropy between two strings of binary such as:

It is primarily when the lengths of two strings is same and in general when the length is not same. and how calculate $p$ and $q$. Is it related to frequency in string or need infer from some more general distribution or what?
For example what is the $q$ and $p$ here?

share|improve this question
Relative entropy is defined between two probability distributions. If you only have samples, you'll need to define a probability distribution and then estimate relative entropy. Is this a homework? –  Memming Nov 5 '13 at 2:46

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.