# Simplyfying Shannon Entropy formula

I need to calculate the following

which is Shannon Entropy formula only using simple functions like log or squareroot or I don't know, just make it simple enough for a guy that does understand only grammar school math, if that is possible, so I don't understand this at all. For starters I don't even know what to do with the "big E" in the beginning of each part of this formula, that is how I "understand" math :(

• The big E (Sigma, $\Sigma$) means sum over all $i$. $\log(1/x) = -\log(x)$ is a well known identity of the logarithm. Would you know how to calculate the entropy given the $x_i$? – mlainz Apr 30 '16 at 4:23
• Or do you want to understand why do we use that formula? – mlainz Apr 30 '16 at 4:25
• The formula looks a little bit scary, but it's actually very simple. – mlainz Apr 30 '16 at 4:38

For example, let $b = 2$. You have 3 events $x_1, x_2, x_3$,
$$p(x_1) = 0.2\\ p(x_2) = 0.5 p(x_3) = 0.3$$
Then $$H = -\sum_i p(x_i) \log_2(p(x_i)) = - p(x_1) \log_2(p(x_1))- p(x_2) \log_2(p(x_2))- p(x_3) \log_2(p(x_3)) =\\ 0.46 + 0.50 + 0.52 = 1.48$$
• Ok. For binary systems you always use $b = 2$. It might be worth to check wikipedia page on password strength – mlainz Apr 30 '16 at 4:42