6
$\begingroup$

It is well known that $E(X^2) \geq (EX)^2$, but I was wondering if there is a similar result for variances, e.g. is $Var(X^2) \geq (VarX)^2$?

I was doing some research and came up with that inequality, but I can’t prove it. I’ve done simulations in R for several known distributions (e.g. bernoulli, binomial, poisson, normal, gamma, t, exponential, all for few parameters) and they seem to show that it really does hold, but I’m not sure whether it holds generally.

If this doesn’t hold, is there any other result that somehow links $Var(X^2)$ and $(VarX)^2$?

EDIT: So, as H. H. Rugh showed in his answer, it can't hold generally, but it often does. I'm still interested in a reference to a different result that gives a certain link between the two expressions, or perhaps some sufficient conditions for the inequality, etc.

$\endgroup$
0

4 Answers 4

7
$\begingroup$

Nice question. It can't hold in general, since you may take a r.v. with values in e.g. $\pm 1$, for which $\text{var}(X^2)=0 < \text{var}(X)^2$.

But numerically it seems to hold when $X$ is sign-definite, e.g. $X\geq 0$ a.s. (in accordance with your simulations).

$\endgroup$
5
$\begingroup$

Let me prove H. H. Rugh's prediction that the inequality is true if

$$ \mathsf{P}(X \geq 0) = 1 \qquad \text{and}\qquad \mathsf{E}[X^4] < \infty. $$

Let $Y \stackrel{d}{=} X$ be an independent copy of $X$. Then notice that

$$ 2\mathsf{Var}(X) = \mathsf{E}[(X - Y)^2] \qquad \text{and} \qquad 2\mathsf{Var}(X^2) = \mathsf{E}[(X^2-Y^2)^2]. $$

Now since $X+Y \geq |X-Y|$, if we set $Z = (X-Y)^2$ then

$$ 2\mathsf{Var}(X^2) = \mathsf{E}[(X-Y)^2(X+Y)^2] \geq \mathsf{E}[Z^2] \geq \mathsf{E}[Z]^2 = 4\mathsf{Var}(X)^2. $$

This actually proves a stronger inequality $ \mathsf{Var}(X^2) \geq 2\mathsf{Var}(X)^2$.


Addendum. Here is an optimal result of this kind:

Claim. If $X \geq 0$ a.s. and $\mathsf{E}[X^4] < \infty$, then we have

$$\mathsf{Var}(X^2) \geq 4 \mathsf{Var}(X)^2.$$

The equality holds if and only if $X$ is either constant or a multiple of the Bernoulli distribution of parameter $\frac{1}{2}$.

Proof. Let $\mu = \mathsf{E}X = \mathsf{E}Y$ denote the common expectation of $X$ and $Y$. Following the previous computation, we find that

\begin{align*} \mathsf{Var}(X^2) &= \frac{1}{2}\mathsf{E}[(X-Y)^2(X+Y)^2] \geq \frac{1}{2}\mathsf{E}[(X-Y)^4] \tag{1} \\ &= \frac{1}{2}\mathsf{E}[((X-\mu)-(Y-\mu))^4] = \mathsf{E}[(X-\mu)^4] + 3\mathsf{Var}(X)^2 \\ &\geq 4\mathsf{Var}(X)^2. \tag{2} \end{align*}

In order for this to be an equality, we need that $\text{(1)}$ and $\text{(2)}$ becomes equality. To avoid unnecessary complication, assume WLOG that $X$ is non-constant.

At $\text{(1)}$, we must have $(X+Y)^2 = (X-Y)^2$ whenever $X \neq Y$. Equivalently, $XY = 0$ must hold whenever $X \neq Y$. This forces that there are at most one non-zero value that $X$ can attain. Indeed, if there are two disjoint Borel sets $B_1, B_2 \subseteq (0, \infty)$ such that $\mathsf{P}(X \in B_i) > 0$ for $i = 1, 2$, then we must have

$$0 = \mathsf{P}(XY \neq 0, X\neq Y) \geq \mathsf{P}(X \in B_1)\mathsf{P}(Y \in B_2) > 0, $$

a contradiction. So there exists $c > 0$ such that $\mathsf{P}(X = c) = p \in (0, 1)$ and $\mathsf{P}(X = 0) = 1-p$. Then

$$ \mathsf{Var}(X^2) = c^4 p(1-p) \qquad \text{and} \qquad 4\mathsf{Var}(X)^2 = 4c^4 p^2(1-p)^2. $$

For these to coincide, we must have $p = \frac{1}{2}$. ////

$\endgroup$
4
  • $\begingroup$ +1 Nice proof. How did you think of making an independent copy of a random variable? Do you have any other examples, links, of the application of this trick? $\endgroup$
    – Hans
    Dec 22, 2017 at 18:15
  • $\begingroup$ @Hans, Thank you for the upvote :) I was motivated by the proof of Harris-FKG inequality, which tells that if $f, g : D \subseteq \mathbb{R} \to \mathbb{R}$ are increasing and $X$ is a random variable taking values in $D$, we have $$ \mathsf{E}[f(X)g(X)] \geq \mathsf{E}f(X)\mathsf{E}g(X) $$ (assuming existence of all expectation therein). The proof goes by introducing an independent copy $Y$ of $X$ and expanding the expectation $$ \mathsf{E}[(f(X) - f(Y))(g(X) - g(Y))] \geq 0 $$, which itself is true since $(f(x) - f(y))(g(x) - g(y)) \geq 0$. $\endgroup$ Dec 22, 2017 at 18:45
  • $\begingroup$ I see. Thanks. The inequality you refers to can be viewed as a variation of the rearrangement inequality. $\endgroup$
    – Hans
    Dec 22, 2017 at 20:17
  • 1
    $\begingroup$ Very nice argument! $\endgroup$
    – H. H. Rugh
    Dec 23, 2017 at 16:56
4
$\begingroup$

Let $\langle\cdot\rangle$ stands for the expectation of $\cdot$. Let $X=x_0+x$ where $x_0=\langle X\rangle$. Your inequality is equivalent to $$2\langle x^2\rangle^2\le (2x_0)^2\langle x^2\rangle+4x_0\langle x^3\rangle+\langle x^4\rangle. \tag1$$ When $x_0=0$, Cauchy-Schwartz inequality $\langle x^2\rangle^2\le\langle x^4\rangle$ eliminates a lot of random variables for which $x^2$ is close to a positive constant but allows the remaining to satisfy the desired inequality.

However, for a fixed $x$, you may adjust $x_0$ to have the same sign as $\langle x^3\rangle$ and sufficiently large absolute value so that the right hand side of $(1)$ is arbitrarily large so that your desired inequality holds.

$\endgroup$
4
  • $\begingroup$ Is the coefficient on $\langle x^3\rangle$ supposed to be a $4$? $\endgroup$
    – jdods
    Dec 22, 2017 at 10:17
  • 1
    $\begingroup$ @jdods: Yes, you are right. I have corrected the coefficient. Please check. Thank you. $\endgroup$
    – Hans
    Dec 22, 2017 at 16:50
  • $\begingroup$ This is actually interesting because my answer doesn't depends on the third moment, but yours does. I think we are both correct (I hope I haven't made an error), just two different ways to achieve the same result. $\endgroup$
    – jdods
    Dec 22, 2017 at 17:27
  • 1
    $\begingroup$ @jdods: Our inequality (1)'s are equivalent, once you expand your $X=\mu+x$ where $x$ is what I define here. I prefer centered moments. $\endgroup$
    – Hans
    Dec 22, 2017 at 20:07
4
$\begingroup$

For mean zero distributions, the inequality $\text{Var}(X^2)\geq\text{Var}(X)^2$ holds when the distribution is not too "platykurtic". "Most" continuous distributions, i.e. the ones we typically come across in a standard probability or statistics course, are mesokurtic or leptokurtic. A sufficiently platykurtic distribution will break the inequality. Intuitively, this means the distribution is sufficiently "flat" with little to no weight in its tails. A symmetric discrete bimodal distribution does the trick as shown in another answer (it has no tails), so does a mean zero symmetric continuous uniform distribution. Anything with mean zero and an excess kurtosis less than $-1$ works. Excess kurtosis is defined as $$\frac{E[(X-\mu)^4]}{\sigma^4}-3.$$

The moments, e.g. $E(X^n)$, generally determine a distribution (not always, see this for clarification regarding that), and they can be understood as describing the shape of the distribution, i.e. where the mass is concentrated, relatively. We can effectively set the moments as we wish (staying within certain restrictions, e.g. not violating Jensen's inequality, etc.) and we have thus defined a distribution.

In general, for any arbitrary distribution with finite mean and variance, if we consider $\mu=E(X)$ and $\sigma^2=\text{Var}(X)=E[(X-\mu)^2]$ to be parameters of the distribution then breaking the inequality requires $$E(X^4)<\mu^4+2\mu^2\sigma^2+\color{red}{2}\sigma^4. \tag1$$

Note the red $2$ since without it, we have $$\mu^4+2\mu^2\sigma^2+\sigma^4\leq E(X^4).$$

For $\mu=0$, we get that the inequality is broken when $$1\leq \frac{E(X^4)}{\sigma^4} < 2.$$

Note that $\frac{E(X^4)}{\sigma^4}=3$ for the mean zero Normal distribution. Any distribution where this ratio is 3 is called "mesokurtic". A distribution where this ratio is greater than 3 is called "leptokurtic" and is often said to have "more weight in the tails than in the center". When it is less than 3, it is called "platykurtic". I would argue that the latter are more rare in practice, but plenty of examples abound.

Theoretically, to get a distribution that breaks the inequality, just set the mean and variance to zero and one, respectively, and set $E(X^4)$ to anything in $[1,2)$. Then set the other moments to whatever you want, and you have an example. Of course setting the moments does not necessarily help us see what the PDF would look like.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.