I have $X_1,X_2,\dots,X_n$ as random samples from a binomial distribution, with probability function:

$$p_X(x) = Pr(X=x) = {m \choose{n}}\alpha^x(1-\alpha)^{m-x},x=0,1,2,\dots,m$$ where $m$ is given and $\alpha \in (0,1)$ is an unknown parameter.

I want to show that the maximum likelihood estimator of $\alpha$ is given by the sample mean over $m$.

Now I imagine the first thing I must do is, find the maximum likelihood estimator.

I believe this is found by finding the joint distribution for my data, and equating the first derivative to be $0$, is this the right direction?

  • $\begingroup$ Yes, that and factoring everything that does not depend on $\alpha$ in the joint likelihood. $\endgroup$ – Did Aug 21 '14 at 11:02
  • $\begingroup$ @Did Could you verify the answer from Nicolas? I would just like a second opinion. If his first assumption is correct, I can see that all of the following method is correct. $\endgroup$ – Tony Aug 21 '14 at 12:22
  • $\begingroup$ Indeed the answer had some misprints, corrected now. $\endgroup$ – Did Aug 21 '14 at 13:23
  • $\begingroup$ @Did Thank you for that. $\endgroup$ – Tony Aug 23 '14 at 11:10

The likelihood function, assuming the samples are independent, is $$L(\alpha) = \prod_{i=1}^N p_X(x_i)= \prod_{i=1}^N {m \choose {x_i} } \alpha^{x_i} (1-\alpha)^{m-x_i}.$$ The next step is to take the logarithm of the likelihood function $$ \log L(\alpha) = \sum_{i=1}^N\log{m \choose {x_i} }+\sum_{i=1}^Nx_i \log(\alpha) +\sum_{i=1}^N(m-x_i) \log(1-\alpha), $$ and to equate its derivative with respect to $\alpha $ to zero, that is, $$ \frac{\partial \log L(\alpha)}{\partial \alpha}=\frac{1}{\alpha}\sum_{i=1}^N x_i -\frac{1}{1-\alpha}\sum_{i=1}^N (m-x_i)=0. $$ From this we deduce that the MLE is $$\hat \alpha=\frac1N\sum_{i=1}^N \frac{x_i}m.$$

  • $\begingroup$ Why have we assumed $P_X = L(\alpha)$? I don't doubt the result, but I thought that we had the joint pdf, normally I work with IID and just taking $\Pi P_X$ $\endgroup$ – Tony Aug 21 '14 at 11:54
  • $\begingroup$ Sorry, you are right. I edited this answer above. $\endgroup$ – nico Aug 21 '14 at 12:40
  • $\begingroup$ If you really want to be precise, you should also verify that the found $\hat \alpha$ really is a maximum by calculating the second derivative. $\endgroup$ – Aahz Aug 21 '14 at 13:33
  • $\begingroup$ @NicolasThorstensen I forgot to thank you, this was very helpful! $\endgroup$ – Tony Aug 26 '14 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.