# Entropy of Zipf and Zeta Distributions

I was wondering how to show entropy of the zeta distribution.

It is: $$H_\mathrm{zeta}(X) = \sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \log(k^s \zeta(s))$$

The entropy of the zipf distribution is:

$$H_\mathrm{zipf}(X) = \frac{s}{H_{N,s}} \sum_{k=1}^N \frac{\ln(k)}{k^s} + \ln(H_{N,s})$$

The zipf distribution with parameter $N = \infty$ is the zeta distribution.

Then, taking the limit $H_\mathrm{zipf}(X)$ as $N$ goes to $\infty$,

$$\lim_{N \to \infty} \frac{s}{H_{N,s}} \sum_{k=1}^N \frac{\ln(k)}{k^s} + \ln(H_{N,s})$$

$$= \frac{s}{\zeta(s)} \sum_{k=1}^\infty (1/k^s) \ln(k) + \ln(\zeta(s))$$

$$= \sum_{k=1}^\infty (1/k^s) \ln(k^s) (1/\zeta(s)) + \ln(\zeta(s))$$

$$= \sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s) + \ln(\zeta(s))\cdots\text{(1)}$$

Note that I have used the identity: $\lim_{N \to \infty} H_{N,s} = \zeta(s)$

In the last equation, equation (1), I was unable to combine the natural log terms because it seems $\ln(\zeta(s))$ is outside of the summation. How do I proceed?

Thanks.

-

Since $\sum\limits_{k=1}^\infty \frac{1/k^s}{\zeta(s)} =1$, (1) is $$\left(\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s)\right) + \ln(\zeta(s))=\left(\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s)\right) + \left(\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)}\right) \ln(\zeta(s)),$$ that is, as desired, $$\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s\zeta(s)).$$
Thank you. The definition of the zeta function times (1 divided by the zeta function) is equal to 1. Also, $\sum_{i=1}^\infty k_i g_i + \sum_{i=1}^\infty k_i A = \sum_{i=1}^\infty k_i(g_i + A)$ – jrand Aug 27 '12 at 23:25