There is a well known classical result called Cramer - Rao bound: https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93Rao_bound Particularly, it is a lower bound for a variance of any unbiased estimate. The question is about to things:

  1. Is there always an estimate which reaches this bound, i.e. is there always exist $\hat{\theta}$ such that, $var(\hat{\theta}) = \frac{1}{I(\theta)}$, $E(\hat\theta) = \theta$? (any standart regularity conditions are included)

  2. If there exist such a estimate, is it always can be found by Maximal Likelihood Method?(as below any regularity conditions are included)

  1. Generally, the answer is no. When you are talking about all estimators it is a little tricky, as this bound holds for asymptotically unbiased estimators. If you are restricted only to unbiased estimators (i.e., "estimable" parameters), then $1/p$, where $X\sim Ber(1,p)$, is an example of non-estimable function. If your class is broader - then MLE's invariance property can provide you with a good estimator of $1/p$.

  2. Maximum likelihood estimators reach this bound asymptotically. Does MLE always exist? It is also somehow tricky question, as finding an MLE is basically finding a maximum of a function. Such a point (estimator) is not always unique and not necessarily exists. However, by putting some regularity conditions on the likelihood function can simplify the task. As such, many problems of this kind are treated within restricted families of distributions like the exponential family or distributions with estimable parameters.

  • $\begingroup$ It would be better, if you could give me some references and links. $\endgroup$ – Byobe Dec 26 '16 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.