I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific.

Let us assume we have some finite set $S$ of elements and we just want to estimate the cardinality $n$ of $S$. We know an upper bound $N$ for the cardinality. One method might be to sample with replacement and count the number of distinct elements and form one's estimate from this using the fact that $E(\text{number of distinct elements in sample})= n(1-(1-1/n)^x)$, where $x$ is the size of the sample.

How do we compute the Fisher Information for this problem? Ultimately I would like to show a lower bound but this would be great first step.

  • 1
    $\begingroup$ The Cramer-Rao theorem says that the variance of an unbiased estimator is at least as large as the inverse of the Fischer information (subject to some regularity conditions). Have you computed the Fisher information for this problem. If you want to show an estimator is efficient, verify that the likelihood satisfies the regularity conditions, compute the variance of the estimator, compute the Fisher information, and compare the variance with the inverse of the information. If they're equal the estimator is efficient, if not not. Please don't crosspost. $\endgroup$ – deinst Nov 20 '12 at 21:26
  • $\begingroup$ Is this route able to show tight bounds? About Fisher information, I am really not sure how to compute it for this problem. Reading en.wikipedia.org/wiki/Fisher_information what I need is a simple worked example. $\endgroup$ – Willem Nov 20 '12 at 21:39
  • $\begingroup$ This seems to have been crossposted. Please ask on one site only. If you don't get an acceptable answer, then flag a moderator to migrate the question to another site. $\endgroup$ – robjohn Nov 20 '12 at 22:22
  • $\begingroup$ This route may be able to show tight bounds. Another route which may work would be to create an unbiased estimator and use the Rao-Blackwell theorem to show that the estimator could not be improved. I note that you have not even proposed an estimator. I would start by proposing one. Computing the likelihood would also probably be useful. $\endgroup$ – deinst Nov 21 '12 at 2:10
  • $\begingroup$ @robjohn, Sorry about the cross post. I didn't know it was bad. How do I fix that now? $\endgroup$ – Willem Nov 21 '12 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.