8
$\begingroup$

Let $X_1, X_2, ..., X_n$ be a random sample from a distribution with p.d.f., $$f(x;\theta)=\theta^2xe^{-x\theta} ; 0<x<\infty, \theta>0$$ Obtain minimum variance unbiased estimator of $\theta$ and examine whether it is attained?

MY WORK:

Using MLE i have found the estimator for $\theta=\frac{2}{\bar{x}}$ Or as $$X\sim \operatorname{Gamma}(2, \theta)$$So $E(X)=2\theta$, $E(\frac{X}{2})=\theta$ so can I take $\frac {X}{2}$ as unbiased estimator of $\theta$. I'm stuck and confused need some help. Thank u.

$\endgroup$
5
  • 1
    $\begingroup$ You first say the estimator is $\hat{\theta}=2/\bar{x}$ but later that $\hat{\theta}=\bar{x}/2$. Which is it? Note that the first one is biased, the second is not. (I think the first one is wrong. See this.) $\endgroup$ Commented Mar 24, 2011 at 9:33
  • $\begingroup$ @amul28, Hint: what do you know about exponential families, sufficiency and completeness? Also, I'm not sure what whoever wrote the question meant by a minimum variance unbiased estimator being "attained". Are you sure this isn't homework? $\endgroup$
    – cardinal
    Commented Mar 24, 2011 at 12:22
  • $\begingroup$ actually i found this question in one of my pervious year question papers and i am not that good in estimation, so im practicing from previous papers. $\endgroup$
    – amul28
    Commented Mar 24, 2011 at 14:27
  • $\begingroup$ oops, my bad. I got confused with the mean. I got the answer as theta^2/2n using CRLB with Gamma(2,theta). $\endgroup$
    – amul28
    Commented Mar 24, 2011 at 15:41
  • $\begingroup$ Have you missed out some words in the question? Something like 'obtain the Cramér–Rao lower bound on the variance of the unbiased estimator and examine whether it is attained' might make a bit more sense. $\endgroup$
    – onestop
    Commented Mar 24, 2011 at 17:27

1 Answer 1

13
$\begingroup$

If one is familiar with the concepts of sufficiency and completeness, then this problem is not too difficult. Note that $f(x; \theta)$ is the density of a $\Gamma(2, \theta)$ random variable. The gamma distribution falls within the class of the exponential family of distributions, which provides rich statements regarding the construction of uniformly minimum variance unbiased estimators via notions of sufficiency and completeness.

The distribution of a random sample of size $n$ from this distribution is $$ g(x_1,\ldots,x_n; \theta) = \theta^{2n} \exp\Big(-\theta \sum_{i=1}^n x_i + \sum_{i=1}^n \log x_i\Big) $$ which, again, conforms to the exponential family class.

From this we can conclude that $S_n = \sum_{i=1}^n X_i$ is a complete, sufficient statistic for $\theta$. Operationally, this means that if we can find some function $h(S_n)$ that is unbiased for $\theta$, then we know immediately via the Lehmann-Scheffe theorem that $h(S_n)$ is the unique uniformly minimum variance unbiased (UMVU) estimator.

Now, $S_n$ has distribution $\Gamma(2n, \theta)$ by standard properties of the gamma distribution. (This can be easily checked via the moment-generating function.)

Furthermore, straightforward calculus shows that $$ \mathbb{E} S_n^{-1} = \int_0^\infty s^{-1} \frac{\theta^{2n} s^{2n - 1}e^{-\theta s}}{\Gamma(2n)} \,\mathrm{d}s = \frac{\theta}{2n - 1} \>. $$

Hence, $h(S_n) = \frac{2n-1}{S_n}$ is unbiased for $\theta$ and must, therefore, be the UMVU estimator.


Addendum: Using the fact that $\newcommand{\e}{\mathbb{E}}\e S_n^{-2} = \frac{\theta^2}{(2n-1)(2n-2)}$, we conclude that the $\mathbb{V}ar(h(S_n)) = \frac{\theta^2}{2(n-1)}$. On the other hand, the information $I(\theta)$ from a sample of size one is readily computed to be $-\e \frac{\partial^2 \log f}{\partial \theta^2} = 2 \theta^{-2}$ and so the Cramer-Rao lower bound for a sample of size $n$ is $$ \mathrm{CRLB}(\theta) = \frac{1}{n I(\theta)} = \frac{\theta^2}{2n} \> . $$

Hence, $h(S_n)$ does not achieve the bound, though it comes close, and indeed, achieves it asymptotically.

However, if we reparametrize the density by taking $\beta = \theta^{-1}$ so that $$ f(x;\beta) = \beta^{-2} x e^{-x/\beta},\quad x > 0, $$ then the UMVU estimator for $\beta$ can be shown to be $\tilde{h}(S_n) = \frac{S_n}{2 n}$. (Just check that it's unbiased!) The variance of this estimator is $\mathbb{V}ar(\tilde{h}(S_n)) = \frac{\beta^2}{2n}$ and this coincides with the CRLB for $\beta$.

The point of the addendum is that the ability to achieve (or not) the CRLB depends on the particular parametrization used and even when there is a one-to-one correspondence between two unique parametrizations, an unbiased estimator for one may achieve the Cramer-Rao lower bound while the other one does not.

$\endgroup$
4
  • $\begingroup$ every thing seems clear except one, how did you take $S_n=(\sum_{i=1}^n X_i)^{-1}$ as i have taken $S_n=(\bar{X})^{-1}$. Where am i going wrong? $\endgroup$
    – amul28
    Commented Mar 25, 2011 at 2:26
  • 1
    $\begingroup$ First, note that my $S_n = \sum_{i=1}^n X_i$ (no inverse). Since $\bar{X}^{-1} = n S_n^{-1}$, it's clear that there is a one-to-one relationship between the two statistics. I simply chose to work with $S_n$ since it has a well-known distribution that is easy to work with. You could have derived the distribution of $(\bar{X})^{-1}$ directly via transformation of variables and ground through the algebra and calculus and got the same answer. It probably just would have been messier. $\endgroup$
    – cardinal
    Commented Mar 25, 2011 at 2:33
  • 1
    $\begingroup$ @amul28, note also that since the variance of the estimator doesn't attain the CRLB, then you really need the concepts of sufficiency and completeness to do this problem. Just finding the MLE and "fixing it up" to be unbiased won't allow you to conclude what you need to. Being able to recognize that a distribution belongs to the exponential family then becomes an important skill. $\endgroup$
    – cardinal
    Commented Mar 25, 2011 at 2:38
  • $\begingroup$ oh right, I get it. I'll try it again. $\endgroup$
    – amul28
    Commented Mar 25, 2011 at 2:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .