Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am wondering if someone knows of a lower bound on the differential entropy of a mixture of two zero-mean Gaussians:

$$h(X)=-\int_{-\infty}^{\infty} f_X(x)\log f_X(x)dx$$

where

$$f_X(x)=\frac{1-\epsilon}{\sqrt{2\pi}\sigma_1}e^{-x^2/2\sigma_1^2}+\frac{\epsilon}{\sqrt{2\pi}\sigma_2}e^{-x^2/2\sigma_2^2}$$

I've tried the trivial lower bound obtained by replacing $\log f_X(x)$ with $f_X(x)-1$, but it's not tight enough. Any suggestions?

share|improve this question

1 Answer 1

up vote 2 down vote accepted

The entropy is a concave function of the probability distribution of the random variable $X$. In other words, if $p_1$ and $p_2$ are probability distributions, then $H(\epsilon p_1 + (1-\epsilon)p_2) \geq \epsilon H(p_1) + (1-\epsilon)H(p_2)$.

You can apply this to get a basic lower bound on your distribution. In your case, $H(X) \geq (1-\epsilon) \log(2\pi e \sigma_1^2) + \epsilon \log(2\pi e \sigma_2^2)$.

Unfortunately, this bound is also not very tight (except in certain regimes), and its usefulness will depend on where you are applying it.

share|improve this answer
    
Can't believe I forgot that the entropy is a concave function of the probability distribution... thanks for the reminder! This bound is indeed loose, but it seems to work in my problem. :) –  M.B.M. Oct 18 '11 at 5:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.