Let $A \subset \Bbb R$ be some set of $n$ numbers. We define its sample variance as $$\frac {1}{n-1}\sum \limits_{i=1}^n (a_i-\bar a)^2,$$ where $\bar a = \frac {a_1+\cdots+a_n}n$ is the average of $A$. It is a surprising fact that for every $2 \leq k \leq n$, the average of the sample variances of all subsets of $A$ of size $k$ equals the sample variance of $A$.

Put in formulas, the claim becomes $$\frac 1 {\binom n k} \sum_{|A|=k} \frac{1}{k-1} \sum_{i=1}^n \left(a_i-\frac 1k \sum_{a_j\in A}a_j\right)^2 = \frac 1{n-1} \sum_{i=1}^n \left(a_i-\frac{a_1+\cdots+a_n}n \right)^2$$

I verified that these awful sums are indeed equal by brute force, where the only tricks are noting that $\sum\limits_{|A|=k}\;\sum\limits_{i \neq j \in A}x_ix_j = \binom{n-2}{k-2}\sum_{i \neq j \in [n]}x_ix_j$ and likewise $\sum\limits_{|A|=k}\;\sum\limits_{i \in A}x_i^2 = \binom{n-1}{k-1}(x_1^2+\cdots+x_n^2)$.

The content of this theorem seems nice enough so that there ought to be better proofs, avoiding the arithmetic miracles, perhaps using the statistical interpretation of each side. I did not find online mentioning of this equality, though. Any ideas?

  • $\begingroup$ Shouldn't the claim include some squares? You can simplify your proof by reducing it to the case $\bar{a}=0$ (since translating the data leaves all variances unchanged), but whether it will become as elegant as you want doing this I don't know. $\endgroup$ – J.G. Jul 28 '17 at 6:55
  • 1
    $\begingroup$ @J.G. I think that assuming $\bar a=0$ actually complicates the proof by a bit. The averages of the sets $A$ will still be nonzero, and the resulting claim will require noting a further fact coming from expanding $0=(a_1+...+a_n)^2$. $\endgroup$ – Emolga Jul 28 '17 at 7:05
  • $\begingroup$ "equals the sample variance of [what?]" A typo? $\endgroup$ – Michael Hardy Jul 28 '17 at 21:14
  • $\begingroup$ If one were to use all sequences of length $k$ rather than all sets of size $k$ (in sequences, repetitions are allowed; in sets they are not) then one would find that the average of this statistics over all sequences of length $k$ would be $$\frac 1 n \sum_{i=1}^n \left(a_i-\frac{a_1+\cdots+a_n}n \right)^2,$$ because the Bessel-corrected sample variance is an unbiased estimator of the population variance. Thus we have the ratio of the average over sequences and the average over sets. $\qquad$ $\endgroup$ – Michael Hardy Jul 28 '17 at 21:32
  • 1
    $\begingroup$ @MichaelHardy I'm working through a problems book for a course in statistics, and this is an exercise there. $\endgroup$ – Emolga Jul 29 '17 at 17:59

I found a satisfying proof. Using the alternative representation of the sample variance as the average of $\frac 12 (a_i-a_j)^2$ over all pairs $a_i,a_j$ makes the claim become the simple assertion that the average of these averages is the overall average. (Which holds since each of them is of the same size, since we restrict to a fixed subset size.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.