# How do we find first order estimate of the given entropy of the following image

What is the solution for this answer? Should we use log base 2 in the formula?

You can see the pixels of images as the realization of a discrete random variable on sample space $$\Omega = \{0, 1, \ldots, 255\}$$ and probability mass function $$p \colon \Omega \in [0,1]$$ defined as: $$p(x) = \begin{cases} \frac{5}{16} & \text{if } x = 200 \text{ or } x = 210,\\ \frac{3}{16} & \text{if } x = 215 \text{ or } x = 217,\\ 0 & \text{otherwise}. \end{cases}$$ By applying the Shannon's definition of entropy we obtain: \begin{align} H(X) &= -\sum_{i=0}^{255} p(i)\cdot \log_b(p(i))\\ &= -\frac{5}{16}\log_b\left(\frac{5}{16}\right)-\frac{5}{16}\log_b\left(\frac{5}{16}\right)-\frac{3}{16}\log_b\left(\frac{3}{16}\right)-\frac{3}{16}\log_b\left(\frac{3}{16}\right)\\ &= -\frac{1}{8}\left(5\log_b\left(\frac{5}{16}\right)-3\log_b\left(\frac{3}{16}\right)\right)\\ &= -\frac{1}{8}\left(\log_b\left(\frac{5^5}{16^5}\frac{3^3}{16^3}\right)\right)\\ &= \frac{1}{8}\left(\log_b\left(\frac{16^8}{5^5 3^3}\right)\right)\\ &= \log_b\left(\frac{16}{\sqrt[8]{5^5 3^3}}\right)\\ \end{align} The choice of the value for the basis $$b$$ of logarithm depends on the units in which you want to express the result:
• base $$2$$: $$H(X) \approx 1.954$$ bits,
• base $$e$$: $$H(X) \approx 1.355$$ nats,
• base $$3$$: $$H(X) \approx 1.233$$ trits,
• base $$10$$: $$H(X) \approx 0.588$$ dits.