Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I know that for a given logmean and logstdev its the lognormal, but what about where we directly specify the mean and variance? The above seems to depend on the log-transformation to the maxent for unbounded continuous RV with given mean and variance (i.e, Normal).

share|improve this question
    
I take the question to mean: what probability distribution of a random variable $X$ with a specified expected value and specified variance has the highest entropy, defined as $\mathbb E(-\log(f(X)))$, where $f$ is the density function? The answer is that $X$ is normally distributed. But at this moment I'd have to do a fair amount of work to write a proof, although I suspect after that I could write a good sketch of it that's much shorter than what I'd need to do first. –  Michael Hardy Dec 8 '13 at 20:15
    
PS: Sorry---I just noticed the restriction to $[0,\infty)$. Maybe I'll be back..... –  Michael Hardy Dec 8 '13 at 20:20
    
...at this point I'm somewhat wildly guesing it's a Gamma distribution. –  Michael Hardy Dec 8 '13 at 20:21
    
@MichaelHardy That's interesting you mention the gamma, because I derived the gamma as the appropriate distribution for the above constraints, but using a different functional other than the Shannon differential entropy. Just wanted to know if entropy will give the same distribution...I haven't seen anything on this in the literature. –  Eupraxis1981 Dec 9 '13 at 0:39

1 Answer 1

up vote 2 down vote accepted

The maximum entropy distribution is of the form $f(x) = \exp( \sum_k \lambda_k g_k(x))/Z$ where $g_k(x)$ are imposed by the restrictions (like Lagrange multipliers) and $Z$ is the normalization factor. In our case, we have two restrictions (apart from the trivial one), which give the two functions $g_1(x)=x$ (mean) and $g_2(x)=x^2$ (second moment, or variance)

Hence, the distribution is has the form $f(x) = \exp( a x -b x^2)/Z$ ($x\ge 0$ , $b>0$) which corresponds to a truncated normal.

share|improve this answer
    
Interesting. Does the positive-unboundedness condition get accounted for when determining the coefficients in the exponent? –  Eupraxis1981 Dec 9 '13 at 5:31
    
Yes, you can check it for the "only mean given" case (you get the exponential distribution). See eg here or here –  leonbloy Dec 9 '13 at 11:56
    
What happens if you specify that the mean and variance have to equal 1? In that case, wouldn't the max entropy distribution still be the exponential, since it is the optimal answer to a problem with one fewer constraints (i.e., variance-relaxation) while still being an answer to the two-constraint problem. A truncated normal whose mean and variance are both one (for the truncated distribution, not the original), would not be an exponential. –  Eupraxis1981 Dec 9 '13 at 16:43
    
@Eupraxis1981 : In that case, we should get $b=0$ (recall that $a,b$ are arbitrary factors that must be adjusted so a to fit the restrictions). "A truncated normal whose mean and variance are both one"... degenerates into an exponential. –  leonbloy Dec 9 '13 at 22:58
    
I see thanks for the help –  Eupraxis1981 Dec 9 '13 at 23:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.