# Maximum Likelihood Estimates Question Using Bernoulli

Suppose that $X$ is a discrete random variable with $P(X = 1) = p$ and $P(X = 2) = 1-p$. Three independent observations of $X$ are made: $x_1 =2, x_2 = 1, x_3 = 2$

a.) Write out likelihood as a function of $p$.

b.) Find the maximum likelihood estimator.

So I first recognized it as a Bernoulli distribution, and got the likelihood function = $3p(1-p)^2$

Then I derived it and found the max which was $p=1/3$. I'm not exactly sure if this approach was correct, can anyone help? Also, if I wanted to check if this was unbiased and consistent, how would I approach doing this? Thanks.

• First part: actually the likelihood is probably rather $p(1-p)^2$ if the observations come in order (but it is difficult to decide from what you write). Second part: to check unbiasedness and consistency, what are your ideas? – Did Aug 15 '15 at 15:13

The likelihood is

$$L(x_1,x_2,x_3\mid p) = \mathbb P(x_1=2)\mathbb P(x_2=1)\mathbb P(x_3=2) = p(1-p)^2.$$

To find the maximum of $L$ we first take the partial derivative with respect to $p$:

$$\frac\partial{\partial p} L(x_1,x_2,x_3\mid p) = (1-p)^2 - 2p(1-p) = (1-p)(1-p-2p) = (1-p)(1-3p).$$

Hence $$\frac\partial{\partial p}L(x_1,x_2,x_3\mid p) = 0 \implies p = 0\text{ or } p = \frac13.$$

We can assume that $p\ne0$ (otherwise the problem is trivial), so we have the candidate MLE $\hat p = \frac13$. Now, $L$ is decreasing on $(\frac13,1)$ (since then $\frac\partial{\partial p}L<0$), and $L$ is concave on $\left(0,\frac23\right)$, as can be seen by

$$\frac{\partial^2}{\partial p^2} L(x_1,x_2,x_3\mid p) = -(1-3p) - 3(1-p) = 2(3p-2).$$ So $\hat p=\frac13$ is the absolute maximum of $L$ on $(0,1)$, and is the maximum likelihood estimator.

Now, $$\mathbb E\left[X\mid p=\frac13\right] = 1\cdot\frac13 + 2\cdot\left(1-\frac13\right) = \frac53,$$ so the bias of $\hat p$ is $$\frac53 - \frac13 = \frac43.$$ Since $\hat p$ is constant with respect to $n$, it's clear that $\hat p$ is not consistent.