I learnt that the variance of the Maximum Likelihood estimator could be calculated as :

\begin{aligned} \operatorname{var}(\theta) &=[I(\theta)]^{-1} \\ &=(-E[H(\theta)])^{-1} \\ &=\left(-E\left[\frac{\partial^{2} \ln \mathcal{L}(\theta)}{\partial \theta \partial \theta^{\prime}}\right]\right)^{-1} \end{aligned}

I want to determine how bias a biased coin is. I've flipped it 3 times and I get 3 Heads.

The likelihood of getting 3 straight heads is $\mathcal{L} = \theta^3$ where $0 \leq \theta \leq 1$

$ \ln \mathcal{L} = 3 \ln \theta $

This means that my $p_{MLE} =1.0$ where $p$ is the probability of getting a heads in a single coin flip and represents the $\theta$ that I'm trying to estimate.

Now obviously because I've only flipped it 3 times, the variance $\text{Var}(p_{MLE})$ will be relatively high. However, what is the actual numerical value of this variance?

I want to show that if I keep flipping the coin and I keep getting heads that this variance goes to zero.

From a similar old post it was found that this variance is equal to:


However I can't see how I can get a numerical answer for the variance of $p_{MLE}$ when we get three straight heads. I know that n=3 but for $p$ what value am I supposed to substitute it with?


$p_{MLE}$ is a random variable. More specifically, here $$p_{MLE} = p_{MLE} (X_1,X_2,X_3) = \frac{1}{3} (X_1 + X_2 + X_3)$$ where $X_i$'s are bernoulli random variables. In the sense in which you're defining your problem, there is no such thing as the variance of $p_{MLE}$ when you get three straight heads, because when you give the value of $p_{MLE}$ for 3 straight heads, it becomes a value rather than a random variable, so its variance is not defined.

  • $\begingroup$ In my case $p_{MLE}$ is the value of p that maximises the likelihood. Now it is possible to get the variance of MLE estimate. See page 7/8 of sherrytowers.com/mle_introduction.pdf $\endgroup$ – piccolo Aug 14 '19 at 6:41
  • $\begingroup$ I want to get some numerical value of the confidence of $p_{MLE}$ which is related to the curvature of the likelihood about $p= p_{MLE}$ $\endgroup$ – piccolo Aug 14 '19 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.