Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

The probability density function of the exponential distribution is defined as

$$ f(x;\lambda)=\begin{cases} \lambda e^{-\lambda x} &\text{if } x \geq 0 \\ 0 & \text{if } x<0 \end{cases} $$

Its likelihood function is

$$ \mathcal{L}(\lambda,x_1,\dots,x_n)=\prod_{i=1}^n f(x_i,\lambda)=\prod_{i=1}^n \lambda e^{-\lambda x}=\lambda^ne^{-\lambda\sum_{i=1}^nx_i} $$

To calculate the maximum likelihood estimation I solved the equation

$$ \frac{d\ln\left(\mathcal{L}(\lambda,x_1,\dots,x_n)\right)}{d\lambda}\overset{!}{=}0 $$

for $\lambda$.

$$ \begin{align} \frac{d\ln\left(\mathcal{L}(\lambda,x_1,\dots,x_n)\right)}{d\lambda} &= \frac{d\ln\left(\lambda^ne^{-\lambda\sum_{i=1}^nx_i}\right)}{d\lambda} \\ &= \frac{d\ln\left(n\ln(\lambda)-\lambda\sum_{i=1}^n x_i\right)}{d\lambda} \\ &= \frac{n}{\lambda}-\sum_{i=1}^n x_i \end{align} $$

Finally we get $$\lambda = \frac{n}{\sum_{i=1}^n x_i}$$

I hope this is correct this far.

Where I am more uncertain is the proof for consistency.

I understand that to be consistent is in this case equivalent to to converge in probability to $\lambda$. So I have a hinch, that something like

$$ \lim_{n\to\infty}\mathbb{P}\left(\mathcal{L}(\lambda,x_1,\dots,x_n)-\lambda\right)=0 $$

will lead me to a solution.

Am I correct this far? If yes, how can I solve this? A hint would be great.


Using Didier Piau's and cardinal's hints I will try to show the consistensy by proving that $\frac{1}{\Lambda_n}\to\frac{1}{\lambda}$ for $n\to\infty$ where

$$ \Lambda_n=\frac{n}{\sum\limits_{k=1}^nX_k} $$

Since $E(X_1)=E\left(\lambda e^{-\lambda x_1}\right)=\frac{1}{\lambda}$ and all $X_i,i=1,2,\dots$ are independent the strong law of great numbers implies that

$$ P\left(\lim\sup_{n\to\infty}\left|\frac{1}{\Lambda_n}-\frac{1}{\lambda}\right|=0\right)=P\left(\lim\sup_{n\to\infty}\left|\frac{\sum\limits_{k=1}^nX_k}{n}-\frac{1}{\lambda}\right|=0\right)=1 $$

is true which implies convergence almost everywhere. This implies convergence in probability of $\Lambda_n$ against $\lambda$ which is equivalent to consistence.

Is this proof correct?

share|improve this question

2 Answers 2

up vote 4 down vote accepted

The computation of the MLE of $\lambda$ is correct.

The consistency is the fact that, if $(X_n)_{n\geqslant1}$ is an i.i.d. sequence of random variables with exponential distribution of parameter $\lambda$, then $\Lambda_n\to\lambda$ in probability, where $\Lambda_n$ denotes the random variable $$ \Lambda_n=\frac{n}{\sum\limits_{k=1}^nX_k}. $$ Thus, one is asked to prove that, for every positive $\varepsilon$, $\mathrm P(|\Lambda_n-\lambda|\geqslant\varepsilon)\to0$ when $n\to\infty$.

In the case at hand, it might be easier to prove the stronger statement that $\frac1{\Lambda_n}\to\frac1\lambda$ almost surely when $n\to\infty$. Hint: Law of large numbers.

share|improve this answer
(+1) I think your answer and my comment were (nearly) simultaneous. I've deleted mine. –  cardinal Jan 23 '12 at 1:30
First, thank you for the explanation. So you mean I have to show, that $P\left(\lim\sup_{n\to\infty}\left|\frac{1}{\Lambda_n}-\frac{1}{\lambda}\right|=‌​0\right)=1$? (Strong law of great numbers.) Correct? –  Aufwind Jan 23 '12 at 1:49
Weak law, if I'm not mistaken. You need to show convergence in probability, not almost sure convergence. –  Michael Hardy Jan 23 '12 at 1:52
Aufwind: Yes, if you know this, you know that $\Lambda_n\to\lambda$ almost surely, hence you know that $\Lambda_n\to\lambda$ in probability, which is what you want. –  Did Jan 23 '12 at 1:53
@DidierPiau I tried another approach. Would you say, that is sufficient? Thanks for any of your efforts! –  Aufwind Jan 23 '12 at 5:55

$\hat\lambda= \frac{n}{\sum_{i=1}^n x_i}$ to be consistent estimator of $\lambda$ it should be Asymptotically

  1. Unbiased,
  2. and it's variance goes to zero.

Using $E\left\{ x\right\}=\frac{1}{\lambda}$ and $E\left\{ x^2\right\}=\frac{2}{\lambda^2}$ and the fact that $x_i$ are iid, we have

Condition 1: $\lim_{n\rightarrow \infty} E\{\hat\lambda - \lambda\}=0$ $$ \begin{aligned} E\left\{ \frac{n}{\sum_{i=1}^n x_i} \right\} - \lambda= \frac{n}{E\left\{\sum_{i=1}^n x_i \right\}} - \lambda= \frac{n}{nE\left\{ x\right\}} - \lambda= \frac{n}{nE\left\{ x\right\}} - \lambda= \lambda - \lambda=0 \end{aligned} $$

Condition 2: $\lim_{n\rightarrow \infty}E\left\{\left(\hat\lambda - E\{\hat\lambda\}\right)^2\right\}=0 $ $$ \begin{aligned} E\left\{\left(\hat\lambda - E\{\hat\lambda\}\right)^2\right\}&=E\left\{\hat\lambda^2\right\} - E\left\{\hat\lambda\right\}^2=E\left\{\frac{n^2}{(\sum_{i=1}^n x_i)(\sum_{i=1}^n x_i)}\right\} -\lambda^2 \\ &= E\left\{\frac{n^2}{\sum_{i,j=1}^n x_ix_j}\right\} -\lambda^2= E\left\{\frac{n^2}{\sum_{i=j} x_ix_j +\sum_{i\neq j}^n x_ix_j}\right\} -\lambda^2 \\ &= \frac{n^2}{nE\left\{x^2\right\}+ n(n-1) E\left\{x\right\}^2} -\lambda^2 = \frac{n}{\frac{2}{\lambda^2}+ \frac{n-1}{\lambda^2}} -\lambda^2 =\frac{n}{n+1}\lambda^2-\lambda^2\\ &\Rightarrow \lim_{n\rightarrow \infty} \frac{n}{n+1}\lambda^2-\lambda^2 =0 \end{aligned} $$

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.