(1) Let $X_1, X_2, \dots, X_n$ be a random sample from a population with non-negative values. Then show that $\bar X \ge S/\sqrt{n},$ where $S^2 = [\sum_{i=1}^n (X_i - \bar X)^2]/(n-1).$

I have not seen this inequality stated before. It is not difficult to prove directly, but may be implied by some more general result I am overlooking.

(2) Also use this inequality to find a counterexample, showing that the sample mean and variance are not independent for exponential (or beta) data; perhaps use $n = 4$ for simplicity. [of course, $\bar X$ and $S$ are independent for normal data.]


Answer to (1).

Note that $(n-1)S^2 = \sum X_i^2 - \frac{1}{n}(\sum X_i)^2$ follows from the definition of $S^2.$

Also, for nonnegative data, $\sum X_i^2 \le (\sum X_i)^2$ because the RHS contains nonnegative cross-product terms that the LHS does not.

Then $$n(n-1)S^2 = n\sum X_i^2 - (\sum X_i)^2 \le n(\sum X_i)^2- (\sum X_i)^2 = (n-1)n^2\bar X^2,$$ which implies (1).

  • $\begingroup$ Several different proofs were given in this answer, and another answer there cites a proof in a 1957 paper by Katsnelson and Kotz. I doubt that any are simpler than yours, though. $\endgroup$ – r.e.s. Apr 7 '16 at 4:59
  • $\begingroup$ (BTW, you've got that last inequality reversed.) $\endgroup$ – r.e.s. Apr 8 '16 at 1:24
  • $\begingroup$ Glad for the reference, fixed the typo. $\endgroup$ – BruceET Apr 8 '16 at 3:25

For the counterexample: picking basically arbitrary numbers, if $n=4$ and $S \geq 2$ then $\overline{X} \geq 1$. So do you have

$$P(S \geq 2 \wedge \overline{X} < 1)=P(S \geq 2)P(\overline{X} < 1)?$$

If not then you have your counterexample. Note that the left side is zero, so it's enough for both factors on the right side to be positive.

Qualitatively speaking, this argument says that $\overline{X}$ and $S$ are dependent because if $S$ is large then $\overline{X}$ must also be large.

  • $\begingroup$ Both probabilities on the RHS do have positive probabilities. The qualitative argument works nicely for exponential. But for BETA(1/2, 1/2), symmetry implies 0 correlation of $\bar X$ and $S$. $\endgroup$ – BruceET Jul 6 '15 at 6:30
  • $\begingroup$ @BruceTrumbo What I wrote doesn't really care whether there is symmetry, it only uses the inequality you gave, which only depends on nonnegativity. The place it could go wrong for a bounded variable is that one or both of the terms on the right side could be $0$. But then just choose a smaller lower bound on the sample standard deviation and adjust the upper bound on the sample mean accordingly. Eventually the right side will be positive and the left side will still be zero. $\endgroup$ – Ian Jul 6 '15 at 13:02
  • $\begingroup$ I was agreeing with your comment on exponential. BETA(1/2,1/2) also has dependence btw sample mean and SD, and the inequality provides a counterexample to independence, but that is a case with 0 correlation and dependence. Only normal has indep sample mean and SD. $\endgroup$ – BruceET Jul 6 '15 at 16:12

Answer and graphs for (2).

The left panel of the figure shows sample SDs $S$ plotted against sample means $\bar X$ for 5000 samples of size 5 from $Exp(1)$; $\bar X$ is highly correlated with $S$ (sample correlation $r = 0.77$) such that $\bar X$ and $S$ tend to increase together.

More specifically, the red dotted line shows $S/\sqrt{5} = \bar X$, so that no points lie above the the line. It is clear that $P(\bar X < 5\sqrt{5}) > 0$ and $P(S > 25) > 0,$ but $P(\bar X <5\sqrt{5},\; S > 25) = 0,$ so that $\bar X$ and $S$ are not independent. (Also, see the Answer by @Ian for exponential data with $n = 4.$)

enter image description here

The right panel shows a similar plot for 5000 samples of size 5 from $Beta(.5, .5).$ Here, it is clear from symmetry that $\bar X$ and $S$ are uncorrelated (sample $r = 0.007$ is consistent with population $\rho = 0.$) However, an argument based on $\bar X \ge S/\sqrt{5}$ provides counterexamples (in the same way as for the exponential distribution) to show that the sample mean and SD are not independent.

Points representing five observations from $Beta(.5,.5)$ lie within the 5-D unit hypercube and tend to lie near its faces, edges, and vertices. When this hypercube is 'squashed' into 2-D by the mean-SD transformation vertices and some edges of the hypercube remain visible. The 5-D hypercube has 32 corners, and there are 6 'horns' visible in the plot; clockwise from lower left, 'multiplicities' in the mapping of vertices to horns are 1, 5, 10, 10, 5, and 1.

Points near the origin have relatively small $\bar X$ and points near the vertex $(0,0,0,1,1)$ (and 19 others) have relatively large $S.$ By the inequality in (1), there can be no points with both 'small' mean and 'large' SD.

Note: Formal proofs notwithstanding, students are often skeptical that $\bar X$ and $S$ are stochastically independent for normal data even though not functionally independent. This intuitive skepticism is not stupid because precisely normal data are rare in applications. And it is only for normal data that $\bar X$ and $S$ are independent. It is intuitively clear why they are not independent for exponential data; the plot in the right panel shows one pattern that thwarts independence in symmetrical nonnormal distributions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.