Does $Var(X^2) \geq (VarX)^2$ hold?

Question

It is well known that $E(X^2) \geq (EX)^2$, but I was wondering if there is a similar result for variances, e.g. is $Var(X^2) \geq (VarX)^2$?

I was doing some research and came up with that inequality, but I can’t prove it. I’ve done simulations in R for several known distributions (e.g. bernoulli, binomial, poisson, normal, gamma, t, exponential, all for few parameters) and they seem to show that it really does hold, but I’m not sure whether it holds generally.

If this doesn’t hold, is there any other result that somehow links $Var(X^2)$ and $(VarX)^2$?

EDIT: So, as H. H. Rugh showed in his answer, it can't hold generally, but it often does. I'm still interested in a reference to a different result that gives a certain link between the two expressions, or perhaps some sufficient conditions for the inequality, etc.

Answer 1

Nice question. It can't hold in general, since you may take a r.v. with values in e.g. $\pm 1$, for which $\text{var}(X^2)=0 < \text{var}(X)^2$.

But numerically it seems to hold when $X$ is sign-definite, e.g. $X\geq 0$ a.s. (in accordance with your simulations).

Answer 2

Let me prove H. H. Rugh's prediction that the inequality is true if

$$ \mathsf{P}(X \geq 0) = 1 \qquad \text{and}\qquad \mathsf{E}[X^4] < \infty. $$

Let $Y \stackrel{d}{=} X$ be an independent copy of $X$. Then notice that

$$ 2\mathsf{Var}(X) = \mathsf{E}[(X - Y)^2] \qquad \text{and} \qquad 2\mathsf{Var}(X^2) = \mathsf{E}[(X^2-Y^2)^2]. $$

Now since $X+Y \geq |X-Y|$, if we set $Z = (X-Y)^2$ then

$$ 2\mathsf{Var}(X^2) = \mathsf{E}[(X-Y)^2(X+Y)^2] \geq \mathsf{E}[Z^2] \geq \mathsf{E}[Z]^2 = 4\mathsf{Var}(X)^2. $$

This actually proves a stronger inequality $ \mathsf{Var}(X^2) \geq 2\mathsf{Var}(X)^2$.

---

Addendum. Here is an optimal result of this kind:

Claim. If $X \geq 0$ a.s. and $\mathsf{E}[X^4] < \infty$, then we have

$$\mathsf{Var}(X^2) \geq 4 \mathsf{Var}(X)^2.$$

The equality holds if and only if $X$ is either constant or a multiple of the Bernoulli distribution of parameter $\frac{1}{2}$.

Proof. Let $\mu = \mathsf{E}X = \mathsf{E}Y$ denote the common expectation of $X$ and $Y$. Following the previous computation, we find that

\begin{align*} \mathsf{Var}(X^2) &= \frac{1}{2}\mathsf{E}[(X-Y)^2(X+Y)^2] \geq \frac{1}{2}\mathsf{E}[(X-Y)^4] \tag{1} \\ &= \frac{1}{2}\mathsf{E}[((X-\mu)-(Y-\mu))^4] = \mathsf{E}[(X-\mu)^4] + 3\mathsf{Var}(X)^2 \\ &\geq 4\mathsf{Var}(X)^2. \tag{2} \end{align*}

In order for this to be an equality, we need that $\text{(1)}$ and $\text{(2)}$ becomes equality. To avoid unnecessary complication, assume WLOG that $X$ is non-constant.

At $\text{(1)}$, we must have $(X+Y)^2 = (X-Y)^2$ whenever $X \neq Y$. Equivalently, $XY = 0$ must hold whenever $X \neq Y$. This forces that there are at most one non-zero value that $X$ can attain. Indeed, if there are two disjoint Borel sets $B_1, B_2 \subseteq (0, \infty)$ such that $\mathsf{P}(X \in B_i) > 0$ for $i = 1, 2$, then we must have

$$0 = \mathsf{P}(XY \neq 0, X\neq Y) \geq \mathsf{P}(X \in B_1)\mathsf{P}(Y \in B_2) > 0, $$

a contradiction. So there exists $c > 0$ such that $\mathsf{P}(X = c) = p \in (0, 1)$ and $\mathsf{P}(X = 0) = 1-p$. Then

$$ \mathsf{Var}(X^2) = c^4 p(1-p) \qquad \text{and} \qquad 4\mathsf{Var}(X)^2 = 4c^4 p^2(1-p)^2. $$

For these to coincide, we must have $p = \frac{1}{2}$. ////

Answer 3

Let $\langle\cdot\rangle$ stands for the expectation of $\cdot$. Let $X=x_0+x$ where $x_0=\langle X\rangle$. Your inequality is equivalent to $$2\langle x^2\rangle^2\le (2x_0)^2\langle x^2\rangle+4x_0\langle x^3\rangle+\langle x^4\rangle. \tag1$$ When $x_0=0$, Cauchy-Schwartz inequality $\langle x^2\rangle^2\le\langle x^4\rangle$ eliminates a lot of random variables for which $x^2$ is close to a positive constant but allows the remaining to satisfy the desired inequality.

However, for a fixed $x$, you may adjust $x_0$ to have the same sign as $\langle x^3\rangle$ and sufficiently large absolute value so that the right hand side of $(1)$ is arbitrarily large so that your desired inequality holds.

Answer 4

For mean zero distributions, the inequality $\text{Var}(X^2)\geq\text{Var}(X)^2$ holds when the distribution is not too "platykurtic". "Most" continuous distributions, i.e. the ones we typically come across in a standard probability or statistics course, are mesokurtic or leptokurtic. A sufficiently platykurtic distribution will break the inequality. Intuitively, this means the distribution is sufficiently "flat" with little to no weight in its tails. A symmetric discrete bimodal distribution does the trick as shown in another answer (it has no tails), so does a mean zero symmetric continuous uniform distribution. Anything with mean zero and an excess kurtosis less than $-1$ works. Excess kurtosis is defined as $$\frac{E[(X-\mu)^4]}{\sigma^4}-3.$$

The moments, e.g. $E(X^n)$, generally determine a distribution (not always, see this for clarification regarding that), and they can be understood as describing the shape of the distribution, i.e. where the mass is concentrated, relatively. We can effectively set the moments as we wish (staying within certain restrictions, e.g. not violating Jensen's inequality, etc.) and we have thus defined a distribution.

In general, for any arbitrary distribution with finite mean and variance, if we consider $\mu=E(X)$ and $\sigma^2=\text{Var}(X)=E[(X-\mu)^2]$ to be parameters of the distribution then breaking the inequality requires $$E(X^4)<\mu^4+2\mu^2\sigma^2+\color{red}{2}\sigma^4. \tag1$$

Note the red $2$ since without it, we have $$\mu^4+2\mu^2\sigma^2+\sigma^4\leq E(X^4).$$

For $\mu=0$, we get that the inequality is broken when $$1\leq \frac{E(X^4)}{\sigma^4} < 2.$$

Note that $\frac{E(X^4)}{\sigma^4}=3$ for the mean zero Normal distribution. Any distribution where this ratio is 3 is called "mesokurtic". A distribution where this ratio is greater than 3 is called "leptokurtic" and is often said to have "more weight in the tails than in the center". When it is less than 3, it is called "platykurtic". I would argue that the latter are more rare in practice, but plenty of examples abound.

Theoretically, to get a distribution that breaks the inequality, just set the mean and variance to zero and one, respectively, and set $E(X^4)$ to anything in $[1,2)$. Then set the other moments to whatever you want, and you have an example. Of course setting the moments does not necessarily help us see what the PDF would look like.