Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

To show whether an MLE I just found is biased/unbiased, would I need to find the expectation of the answer? Plus would I do this by integrating $\text{MLE} \cdot \text{pdf}$.

My MLE is $ \frac{1}{\bar x} $ I've heard the expectation of this is the same as of the expectation of $ \frac{1}{x} $ question 6 part 2, I differentiated th log likelihood and set to zero to get $ \hat\theta = \frac{n}{sum...} = \frac{1}{\bar x} $

share|cite|improve this question
Do you want to state what pdf is in your case and, maybe, show how you derived the MLE? – Sasha Nov 11 '12 at 6:50
see edit, but looks like you beat me to it. – cheeseman123 Nov 11 '12 at 7:11
Could you nevertheless make your post self-contained? One SHOULD NOT have to refer to the content of the link to understand what is going on. – Did Nov 11 '12 at 8:02
up vote 0 down vote accepted

I take it that you are dealing with the exponential distribution with $$ f_X(x) = \lambda \mathrm{e}^{-\lambda x} [x > 0] $$ Assuming all elements of the sample $\{x_1,x_2,\ldots,x_n\}$ are positive, the log-likelihood reads: $$ n \log \lambda - \lambda \sum_{k=1}^n x_k $$ which attains its maximum exactly at $\lambda = \frac{n}{\sum_{k=1}^n x_k}$.

Now to computation of the expectation of the MLE: $$\begin{eqnarray} \mathbb{E}\left(\frac{n}{X_1+X_2+\cdots+X_n}\right) &=& n \mathbb{E}\left(\frac{1}{X_1+X_2+\cdots+X_n}\right) \\ &=& n \mathbb{E}\left( \int_0^\infty \exp\left(-t(X_1+\cdots+X_n)\right) \mathrm{d}t \right) \\ &=& n \int_0^\infty \mathbb{E}\left( \exp\left(-t(X_1+\cdots+X_n)\right) \right) \mathrm{d}t \\ &\stackrel{\text{indep.}}{=}& n \int_0^\infty \left(\mathbb{E}\left( \exp\left(-t X_1\right) \right)\right)^n \mathrm{d}t \\ &=& n \int_0^\infty \left(\frac{\lambda}{t+\lambda}\right)^n \mathrm{d}t = \left. -\frac{n}{n-1} \frac{\lambda^n}{(t+\lambda)^{n-1}} \right|_{0}^\infty \\ &=& \frac{n}{n-1} \lambda \end{eqnarray} $$ Thus the MLE is biased.

share|cite|improve this answer
Do you mind showing me how you got $$\begin{eqnarray} \\ &\stackrel{\text{indep.}}{=}& n \int_0^\infty \left(\mathbb{E}\left( \exp\left(-t X_1\right) \right)\right)^n \mathrm{d}t \\ \end{eqnarray} $$ – cheeseman123 Nov 11 '12 at 7:21
plus from there to $$\begin{eqnarray} &=& n \int_0^\infty \left(\frac{\lambda}{t+\lambda}\right)^n \mathrm{d}t = \left. -\frac{n}{n-1} \frac{\lambda^n}{(t+\lambda)^{n-1}} \right|_{0}^\infty \\ &=& \frac{n}{n-1} \lambda \end{eqnarray} $$ – cheeseman123 Nov 11 '12 at 7:59
The biasedness alone stems from the convexity inequality saying that the mean of the inverse is (strictly) greater than the inverse of the mean. Of course, in the present case, the exact computation is a bonus. – Did Nov 11 '12 at 8:00
@cheeseman123 Independence of $X_k$ was used here:$$ \mathbb{E}\left(\exp(-t(X_1+\cdots+X_n))\right) = \mathbb{E}\left( \mathrm{e}^{-t X_1} \cdots \mathrm{E}^{-t X_n} \right) \stackrel{\text{indep.}}{=} \mathbb{E}\left( \mathrm{e}^{-t X_1} \right) \cdots \mathbb{E}\left( \mathrm{e}^{-t X_n} \right) \stackrel{\text{i.d.}}{=} \left(\mathbb{E}\left( \mathrm{e}^{-t X_1} \right)\right)^n$$ where the last equality follows because $X_k$ are identically distributed, thus means are equal. – Sasha Nov 11 '12 at 15:16
My previous comment was referring (perhaps too cryptically, sorry) to the fact that the inequality you mention is strict unless the random variable is almost surely constant. – Did Nov 11 '12 at 15:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.