Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am encountering a situation where I cannot calculate exact sum of a seris of logorithms in calculating entropy. Suppose we have a series of numbers $p_i$ and we want to calculate $\sum_ilog(p_i)$, we should multiply $p_i$ together then take the log, or just take the log of each $p_i$ and calculate the summation over them?

The $p_i$ satisfies that $\sum_i p_i = 1$ and each $p_i$ is between 0 and 1.

Things are, we multiply them together, seems I will encounter the loss of precision and calculate the multiplication will accumulate the loss of precision. But when doing log apart, then each log seems to have loss of precisions.

share|improve this question
You would actually want to find $\sum_i p_i\log p_i$, know? In standard texts, they always find $\log$ of each $p_i$ and multiply by $p_i$ and sum it up. But I do not know which is better and why. –  Ashok Oct 1 '12 at 12:44

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.