I'm having trouble understanding the solution of a limit. It involves a formula for measuring certainty of a discrete probability distribution.

Given a set of values $p_j$ which sum up to 1, find the limit $\lim_{\alpha\to 1} (\sum p_j^\alpha)^\frac{1}{\alpha-1}$. The author's solution is as follows:

1) Take the natural logarithm, yielding $\lim_{\alpha\to 1} \frac{1}{\alpha-1}\ln\sum p_j^\alpha$.

2) Take the derivative with respect to $\alpha$ of the numerator and denominator to get $\lim_{\alpha\to 1}\frac{\ln\sum p_j^\alpha}{\alpha-1} = \lim_{\alpha\to 1}\frac{\sum p_j^\alpha \ln p_j}{\sum p_j^\alpha}$.

3) We can then plug in $\alpha$, and since $\sum p_j = 1$, we can reduce to $\sum p_j \ln p_j$.

4) Then we undo the natural log to get $\prod p_j^{p_j}$.

I verified the correctness of the result with a few values on WolframAlpha.

The step I don't understand is step 2. When I try to take the derivative of the numerator, I use the chain rule and get $\frac{d}{d\alpha}\ln\sum p_j^\alpha=\frac{1}{\sum p_j^\alpha}\times\sum\alpha p_j^{\alpha-1}$. This simplifies to give an answer larger than 1, which makes no sense.

How do you get $\frac{d}{d\alpha}\ln\sum p_j^\alpha = \frac{\sum p_j^\alpha \ln p_j}{\sum p_j^\alpha}$?

  • 2
    $\begingroup$ You differentiated with respect to $p_j$, not with respect to $\alpha$. Note $p_j^\alpha = \exp(\alpha\cdot \log p_j)$. Perhaps in that form it's easier to differentiate with respect to $\alpha$. $\endgroup$ – Daniel Fischer Aug 28 '15 at 12:34
  • $\begingroup$ Oh! Thank you. $\frac{d}{dx}p_j^\alpha=p_j^\alpha \ln p_j$. Makes sense now. $\endgroup$ – Nate Glenn Aug 28 '15 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.