Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I was going through random numbers and found that the randomness of certain observations is measured by the entropy as given in here. Here, $p(x_i)$ is the probability that $x_i$ will take place. But if I have fair dice then $p(x_i)$ is $\frac{1}{6}$. So, I am assuming that less is the entropy, better is the randomness. Is my conclusion correct? Also, entropy is calculated in terms of - so while determining the efficiency should I consider this negative sign also or only the positive number? I hope my question is clear. Any help will be appreciated!

share|improve this question

1 Answer 1

up vote 1 down vote accepted

Your conclusion is wrong. Entropy in a measure of randomness it is correct but it increases if the randomness increases. There are many different characterizations to maximize the entropy. For example from all densities with equal variance Gaussian density is the one which maximizes the entropy.

On the other hand given a number of events, uniform distribution maximizes the entropy. This means if you have a fair dice it has the maximum entropy and also maximum randomness. If some events are correlated to the others, then entropy decreases as well as the randomness because you can estimate it using the correlation patterns! Read: http://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.