Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I know that for a given logmean and logstdev its the lognormal, but what about where we directly specify the mean and variance? The above seems to depend on the log-transformation to the maxent for unbounded continuous RV with given mean and variance (i.e, Normal).

share|cite|improve this question
I take the question to mean: what probability distribution of a random variable $X$ with a specified expected value and specified variance has the highest entropy, defined as $\mathbb E(-\log(f(X)))$, where $f$ is the density function? The answer is that $X$ is normally distributed. But at this moment I'd have to do a fair amount of work to write a proof, although I suspect after that I could write a good sketch of it that's much shorter than what I'd need to do first. – Michael Hardy Dec 8 '13 at 20:15
PS: Sorry---I just noticed the restriction to $[0,\infty)$. Maybe I'll be back..... – Michael Hardy Dec 8 '13 at 20:20 this point I'm somewhat wildly guesing it's a Gamma distribution. – Michael Hardy Dec 8 '13 at 20:21
@MichaelHardy That's interesting you mention the gamma, because I derived the gamma as the appropriate distribution for the above constraints, but using a different functional other than the Shannon differential entropy. Just wanted to know if entropy will give the same distribution...I haven't seen anything on this in the literature. – user76844 Dec 9 '13 at 0:39
up vote 3 down vote accepted

The maximum entropy distribution is of the form $f(x) = \exp( \sum_k \lambda_k g_k(x))/Z$ where $g_k(x)$ are imposed by the restrictions (like Lagrange multipliers) and $Z$ is the normalization factor. In our case, we have two restrictions (apart from the trivial one), which give the two functions $g_1(x)=x$ (mean) and $g_2(x)=x^2$ (second moment, or variance)

Hence, the distribution is has the form $f(x) = \exp( a x -b x^2)/Z$ ($x\ge 0$ , $b>0$) which corresponds to a truncated normal.

share|cite|improve this answer
Interesting. Does the positive-unboundedness condition get accounted for when determining the coefficients in the exponent? – user76844 Dec 9 '13 at 5:31
Yes, you can check it for the "only mean given" case (you get the exponential distribution). See eg here or here – leonbloy Dec 9 '13 at 11:56
What happens if you specify that the mean and variance have to equal 1? In that case, wouldn't the max entropy distribution still be the exponential, since it is the optimal answer to a problem with one fewer constraints (i.e., variance-relaxation) while still being an answer to the two-constraint problem. A truncated normal whose mean and variance are both one (for the truncated distribution, not the original), would not be an exponential. – user76844 Dec 9 '13 at 16:43
@Eupraxis1981 : In that case, we should get $b=0$ (recall that $a,b$ are arbitrary factors that must be adjusted so a to fit the restrictions). "A truncated normal whose mean and variance are both one"... degenerates into an exponential. – leonbloy Dec 9 '13 at 22:58
I see thanks for the help – user76844 Dec 9 '13 at 23:19

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.