Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am wondering if someone knows of a lower bound on the differential entropy of a mixture of two zero-mean Gaussians:

$$h(X)=-\int_{-\infty}^{\infty} f_X(x)\log f_X(x)dx$$



I've tried the trivial lower bound obtained by replacing $\log f_X(x)$ with $f_X(x)-1$, but it's not tight enough. Any suggestions?

share|cite|improve this question
up vote 3 down vote accepted

The entropy is a concave function of the probability distribution of the random variable $X$. In other words, if $p_1$ and $p_2$ are probability distributions, then $H(\epsilon p_1 + (1-\epsilon)p_2) \geq \epsilon H(p_1) + (1-\epsilon)H(p_2)$.

You can apply this to get a basic lower bound on your distribution. In your case, $H(X) \geq (1-\epsilon) \log(2\pi e \sigma_1^2) + \epsilon \log(2\pi e \sigma_2^2)$.

Unfortunately, this bound is also not very tight (except in certain regimes), and its usefulness will depend on where you are applying it.

share|cite|improve this answer
Can't believe I forgot that the entropy is a concave function of the probability distribution... thanks for the reminder! This bound is indeed loose, but it seems to work in my problem. :) – M.B.M. Oct 18 '11 at 5:32
Is the differential entropy also a concave function of the probability distribution? – river_06 May 8 '15 at 17:48

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.