Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am wondering whether anyone knows of any any inequalities involving the probability density function of an unknown distribution (as opposed to the cumulative distribution function) and its known variance. (We also assume that the mean is zero.)

Background:

I have an upper bound on Kullback–Leibler divergence from $Z$ to $Y=X+Z$

$$D(p_Z\|p_Y)=\int_{-\infty}^{\infty}p_Z(x)\log\frac{p_Z(x)}{p_Y(x)}dx \leq\epsilon$$

where $Z\sim \mathcal{N}(0,\sigma^2)$. $X$ has a symmetric density and mean zero, but is otherwise unknown, thus I am treating $Y$ as an unknown. I am trying to bound the variance of $Y$ given $\epsilon$. To that end I've considered applying the $\mathcal{L}_1$ distance between the two densities, as per this, half of its square bounds KL divergence from below:

$$\frac{1}{2}\left(\int_{-\infty}^{\infty}|p_Z(x)-p_Y(x)|dx\right)^2 \leq D(p_Z\|p_Y)$$

As I understand, $\mathcal{L}_1$ norm relates to the total variation distance. I do not completely understand why or how, and I posted in a separate question about that after helpful comments to this question. But this question is about the need for the characterizing pdf with variance.

share|improve this question
    
There's no such thing as a "cumulative density function". There are density functions, and there are cumulative distribution functions. But the word "cumulative" contradicts the word "density". –  Michael Hardy Oct 1 '11 at 16:56
    
I am a little confused by your last statement and inequality. First of all, what you've listed is not the total variation norm and second, I believe the inequality you've given is false. Let $Z \sim \mathcal N(0,1)$ and $Y \sim \mathcal N(0,1+\epsilon)$ for small enough $\epsilon > 0$. Then, $p_Z(0) - p_Y(0) > D(p_Z \| p_Y)$. –  cardinal Oct 1 '11 at 18:06
    
Thanks, @cardinal I made a pretty large mistake in the background of the question... now fixed. I've also posted an additional related but separate question about the background. –  M.B.M. Oct 2 '11 at 1:22
    
Thanks for the edits. What you've stated is Pinsker's inequality. –  cardinal Oct 2 '11 at 1:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.