Suppose that $X$ is a discrete random variable with $P(X = 1) = p$ and $P(X = 2) = 1-p$. Three independent observations of $X$ are made: $x_1 =2, x_2 = 1, x_3 = 2$

a.) Write out likelihood as a function of $p$.

b.) Find the maximum likelihood estimator.

So I first recognized it as a Bernoulli distribution, and got the likelihood function = $3p(1-p)^2$

Then I derived it and found the max which was $p=1/3$. I'm not exactly sure if this approach was correct, can anyone help? Also, if I wanted to check if this was unbiased and consistent, how would I approach doing this? Thanks.

  • 1
    $\begingroup$ First part: actually the likelihood is probably rather $p(1-p)^2$ if the observations come in order (but it is difficult to decide from what you write). Second part: to check unbiasedness and consistency, what are your ideas? $\endgroup$ – Did Aug 15 '15 at 15:13

The likelihood is

$$L(x_1,x_2,x_3\mid p) = \mathbb P(x_1=2)\mathbb P(x_2=1)\mathbb P(x_3=2) = p(1-p)^2. $$

To find the maximum of $L$ we first take the partial derivative with respect to $p$:

$$\frac\partial{\partial p} L(x_1,x_2,x_3\mid p) = (1-p)^2 - 2p(1-p) = (1-p)(1-p-2p) = (1-p)(1-3p). $$

Hence $$\frac\partial{\partial p}L(x_1,x_2,x_3\mid p) = 0 \implies p = 0\text{ or } p = \frac13. $$

We can assume that $p\ne0$ (otherwise the problem is trivial), so we have the candidate MLE $\hat p = \frac13$. Now, $L$ is decreasing on $(\frac13,1)$ (since then $\frac\partial{\partial p}L<0$), and $L$ is concave on $\left(0,\frac23\right)$, as can be seen by

$$\frac{\partial^2}{\partial p^2} L(x_1,x_2,x_3\mid p) = -(1-3p) - 3(1-p) = 2(3p-2). $$ So $\hat p=\frac13$ is the absolute maximum of $L$ on $(0,1)$, and is the maximum likelihood estimator.

Now, $$\mathbb E\left[X\mid p=\frac13\right] = 1\cdot\frac13 + 2\cdot\left(1-\frac13\right) = \frac53, $$ so the bias of $\hat p$ is $$\frac53 - \frac13 = \frac43. $$ Since $\hat p$ is constant with respect to $n$, it's clear that $\hat p$ is not consistent.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.