1
$\begingroup$

(This question spawned from how to interpret the variance of a variance?)

For a random variable having a normal distribution with zero mean, we know its sample variance (1.2) and also the variance of its sample variance (0.1) -- the numbers are only for example.

What is the 95% confidence interval (e.g. error bounds) for the true value of the standard deviation?

UPDATE

The 95% confidence interval corresponds to 1.96 standard deviations out. Thus, I think the 95% confidence interval for the true standard deviation would be...

$$ \sqrt{1.2} \pm 1.96\times\sqrt{\sqrt{0.1}} $$

Is that right?

$\endgroup$
3
  • $\begingroup$ Sure, use Chebyshev's inequality. $\endgroup$
    – Batman
    Jul 9 '17 at 14:14
  • $\begingroup$ See update above. $\endgroup$
    – user46688
    Jul 9 '17 at 14:57
  • $\begingroup$ For this particular example, because we have the classical result from chi-square distributed pivot, so perhaps you can just construct the CI based on the chi distribution (knowing the sample size and point estimate will do). Interesting to know how these extra sample moments like this to affect the estimation. $\endgroup$
    – BGM
    Jul 9 '17 at 16:49
1
$\begingroup$

First, the fourth root of the variance of the sample variance is not the standard deviation of the sample standard deviation, so I'd recommend keeping things in terms of the variance and writing that the true variance is $1.2 \pm 1.96\sqrt{0.1}$ After that, what you have is a bit incoherent, but nonetheless correct in a sense for large samples.

For $X_i\sim N(\mu,\sigma^2)$ the sample variance $s^2_n=\frac{1}{n-1}\sum_i (X_i-\bar X)^2$ is distributed like $$s^2_n = \frac{\sigma^2}{n-1}\chi^2_{n-1}$$ so the "variance of the variance" is $$ Var(s_n^2) = \frac{(\sigma^2)^2}{(n-1)^2}Var(\chi^2_{n-1})=2\frac{(\sigma^2)^2}{n-1}.$$

Thus, knowing the variance of the variance estimator is tantamount to knowing the true variance itself. Remember, the variance of the variance estimator is a theoretical and not measured quantity and is unknown. Thus, basing confidence intervals on it is incoherent in general. (And sorry, I realize it was my answer to your other question that confused you on this point... I didn't properly distinguish between known and unknown quantities and shouldn't have made it seem like I was giving a prescription for constructing confidence intervals rather than a description of them.) It's possible that for some more complicated setup with more parameters you could have a situation where the variance of the sample variance is known but the variance itself isn't, but not in this situation, and it's unlikely to happen in a realistic situation.

The other issue here is that the $1.96$ figure you use assumes that the sampling distribution of the estimator is normal. As we've already mentioned, the sampling distribution is $\chi^2$, so this isn't true here. However, for large $n$, $\chi^2_n$ is nearly normal.

So in fact for a large sample you can estimate the variance of the variance as $\frac{2s_n^2}{n-1}$ and then your confidence interval in terms of that will be approximately what you said since the sampling distribution is also nearly normal for large samples.

But here's how to actually construct the confidence intervals:

Recall the sample variance is distributed like $$s^2_n = \frac{\sigma^2}{n-1}\chi^2_{n-1}.$$ So $$ (n-1)\frac{s^2_n}{\sigma^2}\sim \chi^2_{n-1}$$ and we have, with $95\%$ confidence, $$\frac{(n-1)s_n^2}{\chi^2_{n-1}(97.5\%)}< \sigma^2 < \frac{(n-1)s_n^2}{\chi^2_{n-1}(2.5\%)}$$ (in other words, in the long run, in $95\%$ of samples, the true variance $\sigma^2$ will lie in this interval.) Here, $\chi^2_{n-1}(97.5\%)$ is the $97.5\%$ quantile of the $\chi^2_{n-1}$ distribution, which can be computed by any software package or looked up in a table.

One more thing: In your example, you say the mean of the normal distribution is known to be zero. In this case, we can actually do better than the above confidence interval. Instead of using the estimator $s_n^2,$ use $$\hat\sigma^2_n = \frac{1}{n}\sum_i X_i^2$$ and you know that $$ \hat\sigma^2_n \sim \frac{\sigma^2}{n}\chi^2_{n}.$$ Then the same method as above gives a better CI.

$\endgroup$
2
  • $\begingroup$ Thanks @spaceisdarkgreen. About the error of only 1 sqrt... if std dev has units of radians, then its variance is in radians^2, and then, isn't the variance of the variance in units of radians^4? If so, then converting back down to std dev (the other direction) requires two square roots, no? Also, what do you consider "large" for $n$? $\endgroup$
    – user46688
    Jul 9 '17 at 20:07
  • 1
    $\begingroup$ Ahh i missed that you were doing the standard deviation. That adds an extra level of complexity and I'd recommend sticking to the variance. (the variance of the variance is not necessarily the square of the variance of the standard deviation). Per my comment about incoherence here, I would be curious as to how you're getting the 'variance of variance' since it's unobservable. In practice, I'd consider $n>100$ or so large. $\endgroup$ Jul 9 '17 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.