# Cramer-Rao and Efficient Estimators

Let $X_1,X_2,X_3,...,X_n$ be a random sample from the exponential distribution having PDF $f(x;\lambda)= \frac{1}{\lambda}e^\frac{-x}{\lambda}\chi\{x>0\}.$

A) Find the Cramer-Rao lower bound for the variance of unbiased estimators for $\theta = \lambda^2$,

B)Determine k so that $W=k\sum\limits_{i=1}^n X^2_i$ is unbiased for $\theta$. Is W an efficient estimator for $\theta?$ (Recall:$E(X^2_i)= 2\lambda^2$ and $E(X^4_i)=24\lambda^4.)$

This is a homework problem, The notes in class and book only cover C.R.L.B for $f_Y(x;\lambda)$ and comparing it to variance of a given estimator which I understand. I don't see the connection though for using it on the variance of unbiased estimators. Is it just different terminology for the same thing or what is it that i'm missing? Can some one help me get started in the right direction?

Every online source seems to reference Fisher information, haven't cover Fisher yet.

• So i found $\lambda_{MLE}$ then used invariance property(don't really understand this) to find $\lambda^2_{MLE} = \bar{x}^2.$ From there used another formula I found online to get $CRLB(\lambda^2) = \frac{[\frac{d}{d\lambda} \lambda^2]^2}{E[\frac{d}{d\lambda} lnf(x;\lambda)]^2} = \frac{4\lambda^8}{n}.$ That might be right for A) but I hate that I don't think I learned anything. Can someone please tell me if it's right and if so explain what happened? Feb 18, 2015 at 9:20
• The derivation for the Cramer-Rao lower bound is quite standard. You can likely look up the corresponding results from your course's references. Feb 18, 2015 at 9:29
• The connection that I didn't make is that $\lambda = \sqrt{\theta}$ and that we then find the CRLB of exponential distribution having pdf $f(x;\sqrt{\theta})$. From there I would of been able to use the Cramer-Rao as I know it. Feb 18, 2015 at 19:02

Define $$I(\lambda)\equiv E\left(\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2\right)$$ where $l(\boldsymbol{X};\lambda)$ denotes the joint likelihood: $$l(\boldsymbol{X};\lambda)=\prod_i\frac{1}{\lambda}\exp(-X_i/\lambda)=\frac{1}{\lambda^n}\exp\left(-\sum_iX_i/\lambda\right)\implies\log l(\boldsymbol{X};\lambda)=-\frac{S_n}{\lambda}-n\log(\lambda).$$ Here $S_n\equiv\sum_iX_i$. With this, you can compute: $$\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2=\left(\frac{S_n}{\lambda^2}-\frac{n}{\lambda}\right)^2=\frac{1}{\lambda^4}(S_n^2-2\lambda n S_n+\lambda^2n^2).$$ Because of independent sampling, $$E\left[\left(\sum_iX_i\right)^2\right]=n E(X_i^2)+(n^2-n)E(X_i)^2=n(2\lambda^2)+(n^2-n)\lambda^2=(n^2+n)\lambda^2,\\ E\left(\sum_iX_i\right)=n E(X_i)=n\lambda.$$ It follows that $$E\left(\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2\right)=\frac{1}{\lambda^2}(n^2+n-2n^2+n^2)=\frac{n}{\lambda^2}\cdot$$ This $I$ that we have computed is called the Fisher information for $\lambda$ for the joint likelihood $l(\boldsymbol{x};\lambda)$. Now the Cramer-Rao lower bound (aka the Frechet-Darmois-Cramer-Rao lower bound) for estimating $g(\lambda)=\lambda^2$ is given by: $$\frac{[g'(\lambda)]^2}{I(\lambda)}=\boxed{\frac{4\lambda^4}{n}}.$$ This completes (a). For (b), note that $E(X_i^2)=2\lambda^2$ so $k=\frac{1}{2n}$ makes $W=k\sum_iX_i^2$ unbiased for $\theta=\lambda^2$. We compute: $$E(W^2)=k^2E\left[\left(\sum_iX_i^2\right)^2\right]=k^2(nE[X_i^4]+(n^2-n)E(X_i^2)^2)=\lambda^4\left(1+\frac{5}{n}\right)\cdot$$ This implies $$\text{Var}(W)=E(W^2)-E(W)^2=\frac{5\lambda^4}{n}>\frac{4\lambda^4}{n}.$$ The last inequality means $W$ is inefficient for $\lambda^2$.
• (a) The Fisher information for $\lambda$ for the joint likelihood is $n$ times the Fisher information for $\lambda$ for the individual likelihood. The latter is easier to compute.
• (b) In fact, a general result implies that only affine transformations of $\lambda$ can be estimated efficiently. Because $\lambda\mapsto\lambda^2$ is not affine, you can conclude without any computation that $W$ with $k=1/(2n)$ is unbiased but inefficient for $\lambda^2$.
• The Fisher information is part of the computation for the CRLB but they are not the same. Even when the function $g$ above is the identity function, the CRLB is the reciprocal of the Fisher information. If you only know the formula for estimating a parameter directly instead of estimating a function of the parameter, I suggest you do the same approach above but in terms of $\theta$ by first substituting in $\lambda=\sqrt{\theta}$. Feb 18, 2015 at 10:17