Prove an Unbiased Estimator of p. Suppose you have a random variable X with distribution Geom(p), and you would like to estimate p.


*

*Since E(x)=1/p, a natural approach to estimate p is to get a sample from X and then take the reciprocal 1/X. Show that this is NOT an unbiased estimator for p.

*Suppose you take n i.i.d. samples X1, X2,...,Xn, and you let Y be the fraction of these samples that equal 1. Show that Y is an unbiased estimator for p. 
 A: Let your estimator be $1/X$ based on a one-element sample. Then, using the probability mass function of the geometric distribution,
\begin{align*}
\mathbb{E}\left(\frac{1}{X}\right)=\sum_{k=1}^{\infty}(1-p)^{k-1}p\times\frac{1}{k}=\frac{p}{1-p}\sum_{k=1}^{\infty}\frac{(1-p)^k}{k}=\frac{p}{1-p}(-\log p),
\end{align*}
where I used the Taylor expansion $\sum_{k=1}^{\infty}(1-x)^k/k=-\log x$, which is valid for any $x\in(0,1)$. This estimator is biased, since its expected value is not equal to $p$.
As for the second estimator, let, for each $i\in\{1,\ldots,n\}$, $K_i$ be the indicator of whether $X_i=1$. Then,
\begin{align*}
Y=\frac{1}{n}\sum_{i=1}^nK_i
\end{align*}
by construction. Note that $\mathbb{E}(K_i)=\mathbb{E}(\mathbf{1}_{\{X_i=1\}})=\mathbb{P}(X_i=1)=p$. Hence,
\begin{align*}
\mathbb{E}(Y)=\mathbb{E}\left(\frac{1}{n}\sum_{i=1}^n K_i\right)=\frac{1}{n}\sum_{i=1}^n\mathbb{E}(K_i)=\frac{1}{n}\sum_{i=1}^n p=\frac{1}{n}\times np=p.
\end{align*}
Therefore, $Y$ is an unbiased estimator of the parameter $p$.
