When we have a sample of numbers, say

-69, 153, -54, 54, -198, -242, -63, 87, -45, -134, ...

we can calculate an estimation of the variance using the formula

$$\hat \sigma^2=s^2=\frac{\sum_{i=1}^n(x_i-\bar{x})^2}{n-1}$$

now this formula applies to any sequence of numbers, regardless of its underlying distribution. But what if we know that these numbers are sampled from a normal distribution, can we provide a better estimate of the variance? And/or if we know the true mean?

  • $\begingroup$ Define what you mean by "better" for an estimator. $\endgroup$ Feb 7 at 15:45
  • $\begingroup$ @user10354138 If I generated these numbers using some code, I want it to be closer to the variance I used to actually generate them. $\endgroup$ Feb 7 at 15:50
  • 1
    $\begingroup$ Now define "closer". Hint: you might want to look at Estimation theory for the many different ways to measure "closeness". $\endgroup$ Feb 7 at 16:00
  • $\begingroup$ @user10354138 just closer... the difference x_pred - x_true is smaller... $\endgroup$ Feb 7 at 16:03
  • $\begingroup$ You still haven't define what "closer" mean in this context, since x_pred (and hence x_pred - x_true) is a random variable. Does, for example, getting closer with 51% probability count when the other 49% are way off? Define it exactly which number (yes, a single number not a random variable) you are going to (try to) minimise. $\endgroup$ Feb 7 at 16:21

If the sample is known to be iid from a normal distribution with unknown mean and variance then $\hat{\sigma^2}$ is the Uniform Minimum Variance Estimator of $\sigma^2$.
If $\mu$ is known, then replace $\bar{x}$ by $\mu$ and delete the $-1$ in the denominator. That will be the UMVUE in that scenario.
If you don't know $\sigma^2$, but have correct knowledge about what it might be (a prior distribution), then the posterior mean of $\sigma^2$ will be biased but have smaller mean-squared error than the UMVUE.

  • $\begingroup$ Thanks for the answer. Is this also the case for iid numbers from other distributions? Is there nothing we can do with the fact that we know for sure what the underlying distribution is? $\endgroup$ Feb 7 at 16:30
  • $\begingroup$ Look at maximum likelihood estimation. It uses your data and your assumption about the underlying distribution. You don't even need the samples to be independent. $\endgroup$ Feb 7 at 16:52
  • $\begingroup$ For other distributions, other estimators may have smaller mean squared error. $\endgroup$
    – John L
    Feb 7 at 17:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.