1
$\begingroup$

Consider a discrete random variable $X$ with the binomial distribution $b(n,p)$ where $n$ is the number of Bernoulli trials and $p\in(0,1)$ is the success probability. The probability mass function for $X$, i.e., the number of successes in $n$ trials, is given by $$ P(X=x)={n \choose x} p^x(1-p)^{n-x},\quad x=0,1,\ldots,n$$

The maximum likelihood estimate (MLE) for $p$ is given by $$\widehat{p} = \frac{x}{n}$$ if one observes the event $X=x$.

My questions are the following:

  1. Can we compute the MLE for $1/p$ as follows: $$\widehat{\frac{1}{p}} = \frac{n}{x}$$ using the invariance property of the MLE?

  2. It seems that the above estimator has infinite mean and variance for any finite $n$ since we have $X=0$ with probability $(1-p)^n$. Does this disturb asymptotic consistency, unbiasedness, and efficiency properties of the MLE?

$\endgroup$
  • $\begingroup$ I solved the problem for you, pls, see the EDIT. $\endgroup$ – zoli Feb 3 '17 at 11:54
2
$\begingroup$

So, suppose that we are Martians and know nothing about the binomial distribution; we know only that we have a parameter $q\geq 1$ and a formula describing the following probabilities

$$P(X=i)=\binom niq^{-i}\left(1-\frac1q\right)^{n-i}.\tag 1$$ ($i=0,1,\cdots, n.$) Now, assume that the outcome of our experiment is $X=0$.

Surprisingly, we are familiar with the maximum likelihood method. So, we apply it. We have to find the $q$ that maximizes

$$\left(1-\frac1q\right)^n.$$

Apparently, for any finite $q$ there is a better one. That is $q=\infty$ seems to be the maximum likelihood estimate.

Now, we suddenly learn what the binomial distribution is. We immediately conclude that $p=0$ is the solution for the "true earthly parameter." Away we sail then immediately.


EDIT

Let's try to find the maximum likelihood parameter $q\geq1$ in the case of $n$ experiments and $i$ successful outcomes assuming that the distribution is given by $(1)$. We can forget about the multiplier $\binom ni$. So, after dividing $(1)$ by $\binom ni$ take the derivative of $(1)$ with respect to $q$. And set the derivative equal to zero then solve the equation for $q$.

Here is the equation

$$(n-i)q^{-i-2}\left(1-\frac1q\right)^{n-i-1}=iq^{-i-1}\left(1-\frac1q\right)^{n-i}.$$

We will have to exclude $q=1$ from now on. However $q=1$ is certainly the solution for $n=i$. Divide both sides by $q^{-i-1}\left(1-\frac1q\right)^{n-i}$. The resulting equation is

$$(n-i)q^{-1}\left(1-\frac1q\right)^{-1}=i.$$

from here we get the expected result:

$$\hat q=\frac ni.$$

NOTE

You can see here that the MLE does have the invariance property. So it is true that if $\frac in$ is the MLE for $p$ then for $q=\frac1p$ the MLE is $\frac ni$. I did the proof above for you and I because I don't believe if theorems (invariance property this time) whose proof I've never digested.

$\endgroup$
  • $\begingroup$ Then, $n/x$ seems to be a valid MLE estimate for $1/p$. But, it has infinite variance and mean for any finite $n$. Does this contradict with asymptotic consistency, unbiasedness, and efficiency properties of the MLE? $\endgroup$ – beaurdeux Feb 2 '17 at 14:01
  • $\begingroup$ @beaurdeux: Yes, it seem. But what is the reason behind this question? $\endgroup$ – zoli Feb 2 '17 at 15:43
  • $\begingroup$ I would like to understand whether the favourable properties (asymptotic unbiasedness, efficiency, and consistency) of the ML estimator does or does not hold for this elementary example? $\endgroup$ – beaurdeux Feb 2 '17 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.