# Cramer-Rao and Efficient Estimators

Let $X_1,X_2,X_3,...,X_n$ be a random sample from the exponential distribution having PDF $f(x;\lambda)= \frac{1}{\lambda}e^\frac{-x}{\lambda}\chi\{x>0\}.$

A) Find the Cramer-Rao lower bound for the variance of unbiased estimators for $\theta = \lambda^2$,

B)Determine k so that $W=k\sum\limits_{i=1}^n X^2_i$ is unbiased for $\theta$. Is W an efficient estimator for $\theta?$ (Recall:$E(X^2_i)= 2\lambda^2$ and $E(X^4_i)=24\lambda^4.)$

This is a homework problem, The notes in class and book only cover C.R.L.B for $f_Y(x;\lambda)$ and comparing it to variance of a given estimator which I understand. I don't see the connection though for using it on the variance of unbiased estimators. Is it just different terminology for the same thing or what is it that i'm missing? Can some one help me get started in the right direction?

Every online source seems to reference Fisher information, haven't cover Fisher yet.

• So i found $\lambda_{MLE}$ then used invariance property(don't really understand this) to find $\lambda^2_{MLE} = \bar{x}^2.$ From there used another formula I found online to get $CRLB(\lambda^2) = \frac{[\frac{d}{d\lambda} \lambda^2]^2}{E[\frac{d}{d\lambda} lnf(x;\lambda)]^2} = \frac{4\lambda^8}{n}.$ That might be right for A) but I hate that I don't think I learned anything. Can someone please tell me if it's right and if so explain what happened? Feb 18, 2015 at 9:20
• The derivation for the Cramer-Rao lower bound is quite standard. You can likely look up the corresponding results from your course's references. Feb 18, 2015 at 9:29
• The connection that I didn't make is that $\lambda = \sqrt{\theta}$ and that we then find the CRLB of exponential distribution having pdf $f(x;\sqrt{\theta})$. From there I would of been able to use the Cramer-Rao as I know it. Feb 18, 2015 at 19:02

## 1 Answer

Define $$I(\lambda)\equiv E\left(\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2\right)$$ where $l(\boldsymbol{X};\lambda)$ denotes the joint likelihood: $$l(\boldsymbol{X};\lambda)=\prod_i\frac{1}{\lambda}\exp(-X_i/\lambda)=\frac{1}{\lambda^n}\exp\left(-\sum_iX_i/\lambda\right)\implies\log l(\boldsymbol{X};\lambda)=-\frac{S_n}{\lambda}-n\log(\lambda).$$ Here $S_n\equiv\sum_iX_i$. With this, you can compute: $$\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2=\left(\frac{S_n}{\lambda^2}-\frac{n}{\lambda}\right)^2=\frac{1}{\lambda^4}(S_n^2-2\lambda n S_n+\lambda^2n^2).$$ Because of independent sampling, $$E\left[\left(\sum_iX_i\right)^2\right]=n E(X_i^2)+(n^2-n)E(X_i)^2=n(2\lambda^2)+(n^2-n)\lambda^2=(n^2+n)\lambda^2,\\ E\left(\sum_iX_i\right)=n E(X_i)=n\lambda.$$ It follows that $$E\left(\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2\right)=\frac{1}{\lambda^2}(n^2+n-2n^2+n^2)=\frac{n}{\lambda^2}\cdot$$ This $I$ that we have computed is called the Fisher information for $\lambda$ for the joint likelihood $l(\boldsymbol{x};\lambda)$. Now the Cramer-Rao lower bound (aka the Frechet-Darmois-Cramer-Rao lower bound) for estimating $g(\lambda)=\lambda^2$ is given by: $$\frac{[g'(\lambda)]^2}{I(\lambda)}=\boxed{\frac{4\lambda^4}{n}}.$$ This completes (a). For (b), note that $E(X_i^2)=2\lambda^2$ so $k=\frac{1}{2n}$ makes $W=k\sum_iX_i^2$ unbiased for $\theta=\lambda^2$. We compute: $$E(W^2)=k^2E\left[\left(\sum_iX_i^2\right)^2\right]=k^2(nE[X_i^4]+(n^2-n)E(X_i^2)^2)=\lambda^4\left(1+\frac{5}{n}\right)\cdot$$ This implies $$\text{Var}(W)=E(W^2)-E(W)^2=\frac{5\lambda^4}{n}>\frac{4\lambda^4}{n}.$$ The last inequality means $W$ is inefficient for $\lambda^2$.

Simplifications:

• (a) The Fisher information for $\lambda$ for the joint likelihood is $n$ times the Fisher information for $\lambda$ for the individual likelihood. The latter is easier to compute.
• (b) In fact, a general result implies that only affine transformations of $\lambda$ can be estimated efficiently. Because $\lambda\mapsto\lambda^2$ is not affine, you can conclude without any computation that $W$ with $k=1/(2n)$ is unbiased but inefficient for $\lambda^2$.
• thanks so much for the help. Quick question though, if the CRLB formula above is standard and delivered the same answer what is the purpose of fisher information? Feb 18, 2015 at 10:02
• never mind. i notice they are the same. essentially i calculated fisher information before with out know the name. Right? Feb 18, 2015 at 10:10
• The Fisher information is part of the computation for the CRLB but they are not the same. Even when the function $g$ above is the identity function, the CRLB is the reciprocal of the Fisher information. If you only know the formula for estimating a parameter directly instead of estimating a function of the parameter, I suggest you do the same approach above but in terms of $\theta$ by first substituting in $\lambda=\sqrt{\theta}$. Feb 18, 2015 at 10:17
• Answering people's homework for them is not only academically highly inappropriate, it also meddles and corrupts the ability of a university to grade. If you would like to help out, post your answer with a 1 or 2 week delay. That way the student still benefits from the answer, but with minimal risk of inappropriate behaviour. Feb 18, 2015 at 10:47
• The student clearly likes to learn hence I helped. The ethical burden is on the OP who poses the question, not the answerers, whose answers the OP could decide to ignore. Plus, the question and the answers will be useful for anyone who at any time later has the same concern. Feb 18, 2015 at 11:38