7
$\begingroup$

Let $X_1,\ldots,X_n$ to be sample distributed geometric with parameter $p$. Find MLE. Is it unbiased?

The distribution for each is $p(1-p)^{x_i-1}$ so the function is $$L(p)=\displaystyle\prod_{i=1}^np(1-p)^{X_i-1}.$$ After taking lns on both sides I got $$l(p)=\ln(L(p))=n\log(p)+\sum_{i=1}^n(X_i-1)\cdot \log(1-p).$$ I derivatied and found maximum in $p_m=\dfrac{n}{n+\sum_{i=1}^n(X_i-1)}$. Now I need to calculate $E[p_m]$: $$E[p_m]=nE\left[\frac{1}{\sum X_i}\right]$$ How can proceed?

$\endgroup$
6
  • $\begingroup$ First note that you expression simplifies, which you can see if you put parentheses where needed: $\sum_i (X_i - 1)$. This is an exponential family, and you will find that the MLE is the same as the method of moments estimator $\hat{p} = 1/\bar{X}$ where $\bar{X} = \frac1n \sum_i X_i$. $\endgroup$
    – passerby51
    Aug 22, 2013 at 0:02
  • $\begingroup$ Now, consider the case $n=1$. Is it true that $E[\frac{1}{X}] = p$? $\endgroup$
    – passerby51
    Aug 22, 2013 at 0:04
  • $\begingroup$ Why $E[\frac 1 X]=p$? $\endgroup$
    – user65985
    Aug 22, 2013 at 0:06
  • 3
    $\begingroup$ For $n=1$, the estimator is $\hat{p} = 1/X_1$. Being unbiased means $E[\hat{p}] = p$. I should have said that as: "Is $E[\frac{1}{X_1}] = p$ true, in which case the estimator is unbiased?" $\endgroup$
    – passerby51
    Aug 22, 2013 at 0:10
  • 1
    $\begingroup$ What I mean is this, when they say an estimator is unbiased, it means that it is unbiased for any number of samples, that is for any $n$. If you can show that it is not unbiased for a particular $n$, the simplest being $n=1$, then you have shown that it is not unbiased. Assume that you just have one sample $X_1$, what is the estimator in this case, and what is its expectation? (You might not be able compute the expectation, but you might be able to show that it is not equal to $p$.) $\endgroup$
    – passerby51
    Aug 22, 2013 at 0:38

1 Answer 1

Reset to default
5
$\begingroup$

Here is one way to answer this. Consider the case $n = 1$. The estimator in this case is $\hat{p} = 1/X_{1}$. Let us try to see what it is expectation is, $$ E[\hat{p}] = E\Big[ \frac1{X_1}\Big] = \sum_{k=1}^\infty \frac{1}{k} P(X_1 = k) = \sum_{k=1}^\infty \frac1k p(1-p)^{k-1} $$

Hint: Note that for $ \alpha \in (-1,1)$, we have $\sum_{k=1}^\infty \frac{\alpha^k}{k} = - \log(1-\alpha)$.

EDIT2: You can obtain the exact expression, or use the following simple bound $$ E(\hat{p}) = p + \sum_{k=2}^{\infty} \frac{1}{k} p (1-p)^{k-1} > p $$ for $p \in (0,1)$ since the sum above is strictly positive.

$\endgroup$
6
  • $\begingroup$ and then $E[\hat{p}]=p\cdot\sum\frac{1}{k}(1-p)^{k-1}< p\cdot\sum(1-p)^{k-1}=\frac{p}{1-(1-p)}=p$ proves the estimation is unbiased? $\endgroup$
    – user65985
    Aug 22, 2013 at 0:51
  • $\begingroup$ Sorry. I didn't check your bound. Your bound does not seem correct. In general bounding is easier, but in this case, the bound you have is $p \sum (1-p)^{k-1} = 1$ which is not enough. $\endgroup$
    – passerby51
    Aug 22, 2013 at 1:15
  • $\begingroup$ fine, another try: $E[\hat{p}]=\frac p {(1-p)} \sum\frac{(1-p)^k}{k}=-\frac{p\cdot log(p)}{1-p}$ which is less than p (since the log for small values of p tend to $-\infty$. Is this fine? $\endgroup$
    – user65985
    Aug 22, 2013 at 1:21
  • 1
    $\begingroup$ For clarification, the MLE of p is then a biased estimator of p? $\endgroup$
    – kathystehl
    Apr 28, 2015 at 18:00
  • 1
    $\begingroup$ @kathystehl, yes it is. $\endgroup$
    – passerby51
    May 1, 2015 at 16:49

You must log in to answer this question.