# how to interpret the variance of a variance?

I understand how to interpret variance, but, what is the physical meaning behind the variance of a variance?

Does it relate to error bounds at all (e.g. confidence interval)?

Perhaps the difficulty is that the word "variance" is being used in two distinct senses in that phrase. The second instance refers to a sample variance or variance estimator. So the more precise phrase is "variance of a variance estimator".

Here's how it works: say you have a sample $x_1,\ldots,x_n$ of data. You can measure the variance of the data in the conventional way by computing $$\hat \sigma^2_n=\frac{1}{n-1}\sum_{i=1}^n (x_i-\bar x)^2.$$

However, you would expect this value you compute to fluctuate from sample to sample. If the true variance were $1$ you might get $1.02$ for one sample and $.99$ for another. You are only going to be really close to the true value if you have a huge sample.

So this sample variance is itself random and as such has a probability distribution. This distribution has a variance defined as $$Var(\hat \sigma^2_n) = E((\hat \sigma^2_n)^2)-E(\hat\sigma^2_n)^2$$ where $E$ is the expectation value. This is a theoretical quantity, not a sample quantity.

You asked about the meaning of this quantity. Roughly speaking, the variance of a random variable tells you how wide its distribution is (well, the square root of the variance does, anyway). So the variance of the sample variance tells you how large of fluctuations in the sample variance we should expect from sample to sample. So if the variance of the sample variance is $0.01$ (whose square root is $0.1$) and our true variance of the distribution the sample comes from is $1$, if we repeatedly took samples and measured their sample variance, we'd expect it to come out to approximately $1\pm 0.1.$ In other words, the random error would have size about $0.1.$

Notice that this quantity depends crucially on the sample size $n.$ For very large samples, we'd expect a very small variance of sample variance, since a large sample will be very representative of the true variance. Likewise if the sample size is small, then there will be large fluctuations in the sample variance. We will have some samples that are uncharacteristically high-variance and some that are uncharacteristically low-variance.

The variance of the sample variance (or of any estimator) relates very much the confidence intervals for that estimator. If you are doing an experiment to measure the the sample variance and you get $1.2$ but you know the variance of the sample variance is only $0.001$, you can be be pretty sure that the true variance is larger than $1,$ i.e. that $1$ is outside your confidence bound. Your confidence bound would be on the order of magnitude of $0.001$ (probably a few times larger).

If you draw a random sample of 10 people from the world population, and measure their heights, would you expect that the sum of the squares of the heights, i.e. $$\sum_{i = 1}^{10} x_i^2$$ to be the same every time you draw such a sample?

Clearly not, because the $x_i$'s will vary between samples. So we can ask what the expected variation of this statistic is, i.e. $$\text{Var}\left( \sum_{i = 1}^{10} x_i^2 \right)$$