# Upper bound for variance of an (arbitrary) zero-mean random variable $X$ given distance between it and a known random variable $Y$

I have a zero-mean Gaussian random variable $Y\sim\mathcal{N}(0,\sigma^2_X)$ with known variance $\sigma_X^2$. I also have a zero-mean random variable $X$, which may be dependent on $Y$ (though, I can tolerate independence assumption if it doesn't hurt the final bound too much). Besides having mean zero, I know two sets of facts about $X$:

1. Its variance, while unknown, is greater than $\sigma_X^2$;
2. One (or more, but one is sufficient) of the following facts hold about the distance between distributions of $X$ and $Y$: \begin{align} D(p_Y\|p_X)&=\int_{-\infty}^{\infty}\frac{\exp(-x^{2}/2\sigma_X^2)}{\sqrt{2\pi}\sigma_X}\log\frac{\exp(-x^{2}/2\sigma_X^2)/\sqrt{2\pi}\sigma_X}{p_X(x)}dx\leq\epsilon_{KL}\\ H^2(Y,X)&=1-\int_{-\infty}^{\infty}\sqrt{\frac{\exp(-x^{2}/2\sigma_X^2)}{\sqrt{2\pi}\sigma_X}p_X(x)}dx\leq\epsilon_{H}\\ TV(Y,X)&=\int_{-\infty}^{\infty}\left|\frac{\exp(-x^{2}/2\sigma_X^2)}{\sqrt{2\pi}\sigma_X}-p_X(x)\right|dx\leq \epsilon_{TV} \end{align}

$D(p_Y\|p_X)$, $H^2(Y,X)$, and $TV(Y,X)$ are Kullback-Leibler divergence, Hellinger distance, and Total variation distance, respectively. These three quantities are commonly used to characterise distance between distributions.

I am trying to find the maximum variance of $X$ such that the distribution for $X$ satisfies any of the above distance requirements. I don't really care about the distribution, just its second moment.

Does anyone have any ideas?

This is a related to a question I asked earlier, but here I've generalised it here quite a bit.

I'm not sure you can bound the variance of $X$ at all. For instance, suppose $X$ is a mixture of distributions which with probability $1-\epsilon$ is $N(0, \sigma_X^2)$, and with probability $\epsilon$ is some distribution with some very large variance $a$. By choosing $\epsilon$ very small, you can at least make the total variation distance small, independent of $a$. Then you can choose $a$ very large so that $X$ has large variance. For the other distances it's not quite as clear what happens, but I invite you to try it.
• Hmmm... you are right about $X$ having an unbounded variance and still satisfying the TV constraint if it's a $(1-\epsilon,\epsilon)$ mixture of $Y$ and some high-variance distribution $Z$. But suppose we had Fact 3 about $X$: $X=Y+Z$ where $Z$ is arbitrary? Can we bound the variance of $X$ (which I guess boils down to bounding variance of $Z$) in that case, given Fact 2? Your answer tells me that I should've included that fact in the original question... – M.B.M. Oct 13 '11 at 12:53
• @Bullmoose: Fact 3 doesn't help. Choose $Z$ to be a mixture which is $0$ with probability $1-\epsilon$ and something with large variance otherwise, and independent of $Y$. Then $Y+Z$ is effectively a mixture of $Y$ and something with large variance. – Nate Eldredge Oct 13 '11 at 13:51