1
$\begingroup$

What kind of distribution do we get if we constrain the range to be the unit interval and also constrain the mean to be $\alpha$?


If we read this table, we see the following two examples of maximum entropy distributions.

The standard normal distribution is the maximum entropy distribution with mean 0 and variance 1.

Similarly, if we restrict out attention to distributions with bounded support, then the uniform distribution is the maximum entropy distribution.

I'm writing a small library to estimate quantiles in a stream of data and am currently keeping track of the sample minimum, maximum, and mean, but am not using the sample mean to estimate quantiles directly.

So, what kind of distribution do we get if we constrain the range to be the unit interval and also constrain the mean to be $\alpha$?

In general, is there a good strategy for figuring out a closed form for the pdf of a maximum entropy distribution satisfying an arbitrary collection of properties?

$\endgroup$
  • 1
    $\begingroup$ For each $\alpha \in (0, 1)$, let $c$ solve the equation $\alpha = 1 - \frac{1}{c} + \frac{1}{e^c - 1}$. (Such $c$ uniquely exists.) Then the distribution with the following p.d.f. $$ f_{\alpha}(x) = \frac{c}{e^c - 1} e^{cx} \mathbf{1}_{[0,1]}(x) $$ maximizes the entropy over the space of continuous distributions supported on $[0, 1]$ and has mean $\alpha$. This follows from variational argument. $\endgroup$ – Sangchul Lee Feb 22 at 23:27
  • $\begingroup$ @SangchulLee, there must be something I'm missing ... but I think that $f_{0.5}$ should be just the uniform distribution since the mean constraint is redundant in the case. $f_{\alpha}$ does not appear to be a constant times an indicator function when $\alpha = 0.5$ . $\endgroup$ – Gregory Nisbet Feb 22 at 23:36
  • 1
    $\begingroup$ That's a good point. As $\alpha \to \frac{1}{2}$, we can prove that $c \to 0$. In such case, $\frac{c}{e^c - 1} \to 1$, so this indeed reduces to the uniform distribution. $\endgroup$ – Sangchul Lee Feb 22 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.