Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am reading a book and having trouble following something. The problem is to try to maximize the differential entropy $-\int_{0}^{\infty}p(r)\log p(r)$ under the constraints that $\int_{0}^{\infty}p(r)=1$ and $\int_{0}^{\infty}rp(r)=\mu_{r}$. The book then just straightaway states that the result is an exponential distribution. I tried to prove this to myself but got stuck, it has been a while since I did any math so I am pretty sure I am missing something very basic.

Here's how far I have gotten. I first wrote the lagrangian

$L=-\int_{0}^{\infty}p(r)\log p(r)+\lambda_{1}\left(\int_{0}^{\infty}p(r)-1\right)+\lambda_{2}\left(\int_{0}^{\infty}rp(r)-\mu_{r}\right)\ \ \ \ \ \ \ \ (1)$

Now here is where I started getting a bit lost, I wasn't sure how to differentiate under the integral so I assumed that the integral was actually a sum over $p(r_{i})$ and differentiated $L$ w.r.t to $p(r_{i})$. Is this correct? What is the proper and general way to solve such differentials?

Moving along, after differentiation (assuming it was a sum) I got the following

$\frac{\partial L}{\partial p}=-\log p(r)-1+\lambda_{1}+\lambda_{2}r=0$

$\implies p(r)=e^{\lambda_{2}r+\lambda_{1}-1}\ \ \ \ \ \ \ \ (2)$

$\frac{\partial L}{\partial\lambda_{1}}=\int_{0}^{\infty}p(r)-1=0$

$\implies\int_{0}^{\infty}e^{\lambda_{2}r+\lambda_{1}-1}dr=1\ \ \ \ \ \ \ \ (3)$

$\frac{\partial L}{\partial\lambda_{2}}=\int_{0}^{\infty}rp(r)-1=0$

$\implies\int_{0}^{\infty}re^{\lambda_{2}r+\lambda_{1}-1}dr=1\ \ \ \ \ \ \ \ (4)$

But now as per my knowledge, the definite integrals in (3) and (4) should equal infinity, but that doesn't help me and so I am stuck? I am sure I am making some very basic conceptual mistake. Can someone please help me with this solution? Thanks.

share|improve this question
    
Somehow, an integral sans differentials looks to me like an unclosed parenthesis... –  J. M. May 6 '11 at 0:29
1  
definite integrals (3) and (4) are not necessarily equal to infinity. Consider $\lambda_2 = -1$ and $\lambda_1 = 1$. –  svenkatr May 6 '11 at 1:53

2 Answers 2

up vote 2 down vote accepted

$\int_0^\infty \mathrm e^{-a x} \mathrm d x = 1/a$ for $a>0$.

$\int_0^\infty r \mathrm e^{-a x} \mathrm d x$ may be evaluated by integration by parts.

share|improve this answer
    
Thanks, I had forgotten my integrals. Just another silly doubt, seeing that I have $e^{\lambda_1 x}$ and not $e^{-\lambda_1 x}$ can I say let $\lambda_1=-c$ and then solve $\int_0^\infty \mathrm e^{-c x}$ and then later substitute back $\lambda_1$? Is there another better method? –  Infinity May 6 '11 at 10:09

Your question about the infinite integrals has already been answered (note that you can make your life easier by absorbing the $1$ into $\lambda_1$), so I'll just answer your question about how to systematically optimize such an integral. The general theory for this sort of thing is the calculus of variations, the main result of which is the Euler-Lagrange equation. However, since your integral depends only on $p$ and not on the derivative of $p$, the Euler-Lagrange equation just reduces to $\frac{\partial \mathcal L}{\partial p}=0$ (where $\mathcal L$ is the Lagrangian density, i.e. the integrand), so your calculation is correct.

share|improve this answer
    
Thanks, I did not know about calculus of variations. –  Infinity May 6 '11 at 10:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.