Let $X_1,\ldots,X_n$ to be sample distributed geometric with parameter $p$. Find MLE. Is it unbiased?

The distribution for each is $p(1-p)^{x_i-1}$ so the function is $$L(p)=\displaystyle\prod_{i=1}^np(1-p)^{X_i-1}.$$ After taking lns on both sides I got $$l(p)=\ln(L(p))=n\log(p)+\sum_{i=1}^n(X_i-1)\cdot \log(1-p).$$ I derivatied and found maximum in $p_m=\dfrac{n}{n+\sum_{i=1}^n(X_i-1)}$. Now I need to calculate $E[p_m]$: $$E[p_m]=nE\left[\frac{1}{\sum X_i}\right]$$ How can proceed?

  • $\begingroup$ First note that you expression simplifies, which you can see if you put parentheses where needed: $\sum_i (X_i - 1)$. This is an exponential family, and you will find that the MLE is the same as the method of moments estimator $\hat{p} = 1/\bar{X}$ where $\bar{X} = \frac1n \sum_i X_i$. $\endgroup$ – passerby51 Aug 22 '13 at 0:02
  • $\begingroup$ Now, consider the case $n=1$. Is it true that $E[\frac{1}{X}] = p$? $\endgroup$ – passerby51 Aug 22 '13 at 0:04
  • $\begingroup$ Why $E[\frac 1 X]=p$? $\endgroup$ – user65985 Aug 22 '13 at 0:06
  • 2
    $\begingroup$ For $n=1$, the estimator is $\hat{p} = 1/X_1$. Being unbiased means $E[\hat{p}] = p$. I should have said that as: "Is $E[\frac{1}{X_1}] = p$ true, in which case the estimator is unbiased?" $\endgroup$ – passerby51 Aug 22 '13 at 0:10
  • $\begingroup$ but as far as I know when we talk about discrete variable )X can get values $x_1... x_k$) $E[X]=\sum_{i=1}^k x*P(X=x_i)$ but here we have various xs. I still don't understand how can we answer the question you asked. $\endgroup$ – user65985 Aug 22 '13 at 0:17

Here is one way to answer this. Consider the case $n = 1$. The estimator in this case is $\hat{p} = 1/X_{1}$. Let us try to see what it is expectation is, $$ E[\hat{p}] = E\Big[ \frac1{X_1}\Big] = \sum_{k=1}^\infty \frac{1}{k} P(X_1 = k) = \sum_{k=1}^\infty \frac1k p(1-p)^{k-1} $$

Hint: Note that for $ \alpha \in (-1,1)$, we have $\sum_{k=1}^\infty \frac{\alpha^k}{k} = - \log(1-\alpha)$.

EDIT2: You can obtain the exact expression, or use the following simple bound $$ E(\hat{p}) = p + \sum_{k=2}^{\infty} \frac{1}{k} p (1-p)^{k-1} > p $$ for $p \in (0,1)$ since the sum above is strictly positive.

  • $\begingroup$ and then $E[\hat{p}]=p\cdot\sum\frac{1}{k}(1-p)^{k-1}< p\cdot\sum(1-p)^{k-1}=\frac{p}{1-(1-p)}=p$ proves the estimation is unbiased? $\endgroup$ – user65985 Aug 22 '13 at 0:51
  • $\begingroup$ Sorry. I didn't check your bound. Your bound does not seem correct. In general bounding is easier, but in this case, the bound you have is $p \sum (1-p)^{k-1} = 1$ which is not enough. $\endgroup$ – passerby51 Aug 22 '13 at 1:15
  • $\begingroup$ fine, another try: $E[\hat{p}]=\frac p {(1-p)} \sum\frac{(1-p)^k}{k}=-\frac{p\cdot log(p)}{1-p}$ which is less than p (since the log for small values of p tend to $-\infty$. Is this fine? $\endgroup$ – user65985 Aug 22 '13 at 1:21
  • $\begingroup$ The expression $- p \log p / (1-p)$ seems correct to me. It is enough to say that this is not equal $p$ for all $p \in (0,1)$, which is obvious. However, it seems that in fact $-p \log(p) /(1-p)$ is $> p$ over $(0,1)$. Here is a link to a plot of $E[\hat{p}] / p$ suggesting this: wolfr.am/14kDWU9 $\endgroup$ – passerby51 Aug 22 '13 at 1:26
  • $\begingroup$ For clarification, the MLE of p is then a biased estimator of p? $\endgroup$ – kathystehl Apr 28 '15 at 18:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy