Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $X_1, X_2, ..., X_n$ be a random sample from a distribution with p.d.f., $$f(x;\theta)=\theta^2xe^{-x\theta} ; 0<x<\infty, \theta>0$$ Obtain minimum variance unbiased estimator of $\theta$ and examine whether it is attained?


Using MLE i have found the estimator for $\theta=\frac{2}{\bar{x}}$ Or as $$X\sim Gamma(2, \theta)$$So $E(X)=2\theta$ $E(\frac{x}{2})=\theta$ so can i take $\frac {x}{2}$ as unbiased estimator of $\theta$. I'm stuck and confused need some help.

Thank u.

share|cite|improve this question
You first say the estimator is $\hat{\theta}=2/\bar{x}$ but later that $\hat{\theta}=\bar{x}/2$. Which is it? Note that the first one is biased, the second is not. (I think the first one is wrong. See this.) – Raskolnikov Mar 24 '11 at 9:33
@amul28, Hint: what do you know about exponential families, sufficiency and completeness? Also, I'm not sure what whoever wrote the question meant by a minimum variance unbiased estimator being "attained". Are you sure this isn't homework? – cardinal Mar 24 '11 at 12:22
actually i found this question in one of my pervious year question papers and i am not that good in estimation, so im practicing from previous papers. – amul28 Mar 24 '11 at 14:27
oops, my bad. I got confused with the mean. I got the answer as theta^2/2n using CRLB with Gamma(2,theta). – amul28 Mar 24 '11 at 15:41
Have you missed out some words in the question? Something like 'obtain the Cramér–Rao lower bound on the variance of the unbiased estimator and examine whether it is attained' might make a bit more sense. – onestop Mar 24 '11 at 17:27
up vote 9 down vote accepted

If one is familiar with the concepts of sufficiency and completeness, then this problem is not too difficult. Note that $f(x; \theta)$ is the density of a $\Gamma(2, \theta)$ random variable. The gamma distribution falls within the class of the exponential family of distributions, which provides rich statements regarding the construction of uniformly minimum variance unbiased estimators via notions of sufficiency and completeness.

The distribution of a random sample of size $n$ from this distribution is $$ g(x_1,\ldots,x_n; \theta) = \theta^{2n} \exp\Big(-\theta \sum_{i=1}^n x_i + \sum_{i=1}^n \log x_i\Big) $$ which, again, conforms to the exponential family class.

From this we can conclude that $S_n = \sum_{i=1}^n X_i$ is a complete, sufficient statistic for $\theta$. Operationally, this means that if we can find some function $h(S_n)$ that is unbiased for $\theta$, then we know immediately via the Lehmann-Scheffe theorem that $h(S_n)$ is the unique uniformly minimum variance unbiased (UMVU) estimator.

Now, $S_n$ has distribution $\Gamma(2n, \theta)$ by standard properties of the gamma distribution. (This can be easily checked via the moment-generating function.)

Furthermore, straightforward calculus shows that $$ \mathbb{E} S_n^{-1} = \int_0^\infty s^{-1} \frac{\theta^{2n} s^{2n - 1}e^{-\theta s}}{\Gamma(2n)} \,\mathrm{d}s = \frac{\theta}{2n - 1} \>. $$

Hence, $h(S_n) = \frac{2n-1}{S_n}$ is unbiased for $\theta$ and must, therefore, be the UMVU estimator.

Addendum: Using the fact that $\newcommand{\e}{\mathbb{E}}\e S_n^{-2} = \frac{\theta^2}{(2n-1)(2n-2)}$, we conclude that the $\mathbb{V}ar(h(S_n)) = \frac{\theta^2}{2(n-1)}$. On the other hand, the information $I(\theta)$ from a sample of size one is readily computed to be $-\e \frac{\partial^2 \log f}{\partial \theta^2} = 2 \theta^{-2}$ and so the Cramer-Rao lower bound for a sample of size $n$ is $$ \mathrm{CRLB}(\theta) = \frac{1}{n I(\theta)} = \frac{\theta^2}{2n} \> . $$

Hence, $h(S_n)$ does not achieve the bound, though it comes close, and indeed, achieves it asymptotically.

However, if we reparametrize the density by taking $\beta = \theta^{-1}$ so that $$ f(x;\beta) = \beta^{-2} x e^{-x/\beta},\quad x > 0, $$ then the UMVU estimator for $\beta$ can be shown to be $\tilde{h}(S_n) = \frac{S_n}{2 n}$. (Just check that it's unbiased!) The variance of this estimator is $\mathbb{V}ar(\tilde{h}(S_n)) = \frac{\beta^2}{2n}$ and this coincides with the CRLB for $\beta$.

The point of the addendum is that the ability to achieve (or not) the CRLB depends on the particular parametrization used and even when there is a one-to-one correspondence between two unique parametrizations, an unbiased estimator for one may achieve the Cramer-Rao lower bound while the other one does not.

share|cite|improve this answer
every thing seems clear except one, how did you take $S_n=(\sum_{i=1}^n X_i)^{-1}$ as i have taken $S_n=(\bar{X})^{-1}$. Where am i going wrong? – amul28 Mar 25 '11 at 2:26
First, note that my $S_n = \sum_{i=1}^n X_i$ (no inverse). Since $\bar{X}^{-1} = n S_n^{-1}$, it's clear that there is a one-to-one relationship between the two statistics. I simply chose to work with $S_n$ since it has a well-known distribution that is easy to work with. You could have derived the distribution of $(\bar{X})^{-1}$ directly via transformation of variables and ground through the algebra and calculus and got the same answer. It probably just would have been messier. – cardinal Mar 25 '11 at 2:33
@amul28, note also that since the variance of the estimator doesn't attain the CRLB, then you really need the concepts of sufficiency and completeness to do this problem. Just finding the MLE and "fixing it up" to be unbiased won't allow you to conclude what you need to. Being able to recognize that a distribution belongs to the exponential family then becomes an important skill. – cardinal Mar 25 '11 at 2:38
oh right, I get it. I'll try it again. – amul28 Mar 25 '11 at 2:47

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.