Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'll preface this by saying that I haven't taken an in depth study of the calculus of variations and have only come across it recently in applications; in depth study is on my to do list.

I'm wondering if there is any sense in which the following heuristic for functional optimization is valid. I'm interested in finding a probability density $f$ on $[0, 1]$ with maximum entropy $H[f] = - \int f \log f \ dx$, subject to the constraint that $\int x f(x) \ dx = \mu$. I set this up via Lagrange multipliers as $$ \arg \max_f \ -\int f \log f \ dx - \lambda_1 \left(\int f \ dx - 1\right) - \lambda_2 \left(\int x f \ dx - \mu\right). $$ Now, instead I solve an approximate problem using Riemann sums; we partition $[0, 1]$ with $0 = x_0 < x_1 < \cdots < x_n = 1$ and approximate this by $$ \arg \max_f \ -\sum f_{x_j} \log f_{x_j} \Delta_{x_j} - \lambda_1 \left(\sum f_{x_j} \Delta_{x_j} - 1\right) - \lambda_2 \left(\sum x_{j} f_{x_j} \ \Delta_{x_j} - \mu \right). $$ This can be solved via multivariate calculus and leads to $f(x) = a e^{bx}$, a truncated exponential distribution, as the density on $[0, 1]$ with maximal entropy for a fixed mean, which I believe is the correct answer.

There are a few problems right off the bat. This makes the assumption of Riemann integrability, which is undesirable. It also ignores measure theoretic details (e.g. the solution to the optimization should only be unique up to a.e. $dx$ equivalence classes). But perhaps if I constrain the optimization to being over continuous functions, this provides an argument which can be patched up and made rigorous?

share|cite|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.