For example, we want to estimate the covariance matrix from a given set of data points. We can derive an unbiased estimator. However, we can still derive an estimator by minimizing information loss in the data.

Then, why is unbiased covariance estimator important regarding statistical estimation and real-world application?

  • $\begingroup$ Can you clarify what you mean by "minimizing information loss"? If you have a parametrized family of distributions you can use maximum likelihood, which generally would be biased for this sort of thing but might perform slightly better in some ways that unbiased estimators. (Certainly in the one-dimensional case it does if you're using squared-error loss. But only slightly.) I wouldn't attach any great importance to unbiasedness in this context. $\endgroup$ – Michael Hardy Apr 15 '17 at 18:19
  • $\begingroup$ The information means the Entropy ttic.uchicago.edu/~dmcallester/ttic101-06/lectures/entropy/… , and we hope that we can keep entropy in the data as much as possible when deriving the estimator. This is almost equivalent to the maximum likelihood cs.huji.ac.il/~shashua/papers/class3-ML-MaxEnt.pdf. As you said, biased estimator can perform slightly better than unbiased, could you please show why we need unbiased in the perspective of the theory analysis and practical application? Please $\endgroup$ – olivia Apr 16 '17 at 2:28
  • $\begingroup$ Without further context I know of no reason why unbiasedness is needed. $\endgroup$ – Michael Hardy Apr 16 '17 at 5:12
  • $\begingroup$ @MichaelHardy I think we need unbiasedness because it allows the estimator to converge the target covariance after receiving more data samples. $\endgroup$ – olivia Apr 16 '17 at 7:07
  • $\begingroup$ You don't need unbiasedness for that. The MLE also converges to the population covariance. $\endgroup$ – Michael Hardy Apr 16 '17 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.