3
$\begingroup$

I have a distribution with literally an infinite number of potential data points. I need the standard deviation. I generate about a hundred points and take the standard deviation of the points. This gives a hopefully good approximation of the true standard deviation, but it won't, of course, be exact. How do I estimate the uncertainty in the standard deviation? This seems like a very basic question, but web searching hasn't provided any solution. If I missed it somehow, my apologies.

$\endgroup$
  • $\begingroup$ What do you mean by uncertainty in the standard deviation? $\endgroup$ – velut luna Apr 30 '15 at 16:44
  • $\begingroup$ @KYson I mean that the distribution has some value of standard deviation. I can get some approximation to that value by pulling a number of points out of it and taking their standard deviation. But the number that I get won't be the exact S.D. of the entire distribution. But surely, I am thinking, it must be good to within some "window". What is "the width of that window"? $\endgroup$ – bob.sacamento Apr 30 '15 at 18:11
  • $\begingroup$ You said about a hundred points. Can use central limit theorem? $\endgroup$ – velut luna Apr 30 '15 at 18:26
  • $\begingroup$ @Kyson I've looked it over. I don't see how I can use it. I'm admittedly not the sharpest tool in the shed sometimes. Any details you are thinking about will be most welcome. $\endgroup$ – bob.sacamento Apr 30 '15 at 20:01
6
$\begingroup$

If you want to find out the uncertainty or standard error (SE) in the standard deviation of a chosen sample, then you can simply use $SE(\sigma) = \frac{\sigma}{\sqrt{2N - 2}}$, where $N$ is the number of data points in your sample.

Hope that helps!

$\endgroup$
3
$\begingroup$

If you're allowed to take that sample repeatedly, it's basically bootstrapping.

Procedure:

  1. Draw 100 points

  2. Calculate standard deviation

  3. Repeat Steps 1 & 2 a lot of times (empirically, I've found 5-10,000 to be enough), keeping track of the results of step 2.

  4. Examine the distribution of estimates from Step 2 with whatever tools you'd like -- histograms, sample moments, etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.