2
$\begingroup$

Let $X_1,X_2,X_3,...,X_n$ be a random sample from the exponential distribution having PDF $f(x;\lambda)= \frac{1}{\lambda}e^\frac{-x}{\lambda}\chi\{x>0\}.$

A) Find the Cramer-Rao lower bound for the variance of unbiased estimators for $\theta = \lambda^2$,

B)Determine k so that $W=k\sum\limits_{i=1}^n X^2_i$ is unbiased for $\theta$. Is W an efficient estimator for $\theta?$ (Recall:$E(X^2_i)= 2\lambda^2$ and $E(X^4_i)=24\lambda^4.)$

This is a homework problem, The notes in class and book only cover C.R.L.B for $f_Y(x;\lambda)$ and comparing it to variance of a given estimator which I understand. I don't see the connection though for using it on the variance of unbiased estimators. Is it just different terminology for the same thing or what is it that i'm missing? Can some one help me get started in the right direction?

Every online source seems to reference Fisher information, haven't cover Fisher yet.

$\endgroup$
3
  • $\begingroup$ So i found $\lambda_{MLE}$ then used invariance property(don't really understand this) to find $\lambda^2_{MLE} = \bar{x}^2.$ From there used another formula I found online to get $CRLB(\lambda^2) = \frac{[\frac{d}{d\lambda} \lambda^2]^2}{E[\frac{d}{d\lambda} lnf(x;\lambda)]^2} = \frac{4\lambda^8}{n}.$ That might be right for A) but I hate that I don't think I learned anything. Can someone please tell me if it's right and if so explain what happened? $\endgroup$
    – Damir
    Feb 18, 2015 at 9:20
  • $\begingroup$ The derivation for the Cramer-Rao lower bound is quite standard. You can likely look up the corresponding results from your course's references. $\endgroup$ Feb 18, 2015 at 9:29
  • $\begingroup$ The connection that I didn't make is that $\lambda = \sqrt{\theta}$ and that we then find the CRLB of exponential distribution having pdf $f(x;\sqrt{\theta})$. From there I would of been able to use the Cramer-Rao as I know it. $\endgroup$
    – Damir
    Feb 18, 2015 at 19:02

1 Answer 1

3
$\begingroup$

Define $$ I(\lambda)\equiv E\left(\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2\right) $$ where $l(\boldsymbol{X};\lambda)$ denotes the joint likelihood: $$ l(\boldsymbol{X};\lambda)=\prod_i\frac{1}{\lambda}\exp(-X_i/\lambda)=\frac{1}{\lambda^n}\exp\left(-\sum_iX_i/\lambda\right)\implies\log l(\boldsymbol{X};\lambda)=-\frac{S_n}{\lambda}-n\log(\lambda). $$ Here $S_n\equiv\sum_iX_i$. With this, you can compute: $$ \left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2=\left(\frac{S_n}{\lambda^2}-\frac{n}{\lambda}\right)^2=\frac{1}{\lambda^4}(S_n^2-2\lambda n S_n+\lambda^2n^2). $$ Because of independent sampling, $$ E\left[\left(\sum_iX_i\right)^2\right]=n E(X_i^2)+(n^2-n)E(X_i)^2=n(2\lambda^2)+(n^2-n)\lambda^2=(n^2+n)\lambda^2,\\ E\left(\sum_iX_i\right)=n E(X_i)=n\lambda. $$ It follows that $$ E\left(\left[\frac{\partial\log l(\boldsymbol{X};\lambda)}{\partial\lambda}\right]^2\right)=\frac{1}{\lambda^2}(n^2+n-2n^2+n^2)=\frac{n}{\lambda^2}\cdot $$ This $I$ that we have computed is called the Fisher information for $\lambda$ for the joint likelihood $l(\boldsymbol{x};\lambda)$. Now the Cramer-Rao lower bound (aka the Frechet-Darmois-Cramer-Rao lower bound) for estimating $g(\lambda)=\lambda^2$ is given by: $$ \frac{[g'(\lambda)]^2}{I(\lambda)}=\boxed{\frac{4\lambda^4}{n}}. $$ This completes (a). For (b), note that $E(X_i^2)=2\lambda^2$ so $k=\frac{1}{2n}$ makes $W=k\sum_iX_i^2$ unbiased for $\theta=\lambda^2$. We compute: $$ E(W^2)=k^2E\left[\left(\sum_iX_i^2\right)^2\right]=k^2(nE[X_i^4]+(n^2-n)E(X_i^2)^2)=\lambda^4\left(1+\frac{5}{n}\right)\cdot $$ This implies $$ \text{Var}(W)=E(W^2)-E(W)^2=\frac{5\lambda^4}{n}>\frac{4\lambda^4}{n}. $$ The last inequality means $W$ is inefficient for $\lambda^2$.


Simplifications:

  • (a) The Fisher information for $\lambda$ for the joint likelihood is $n$ times the Fisher information for $\lambda$ for the individual likelihood. The latter is easier to compute.
  • (b) In fact, a general result implies that only affine transformations of $\lambda$ can be estimated efficiently. Because $\lambda\mapsto\lambda^2$ is not affine, you can conclude without any computation that $W$ with $k=1/(2n)$ is unbiased but inefficient for $\lambda^2$.
$\endgroup$
7
  • $\begingroup$ thanks so much for the help. Quick question though, if the CRLB formula above is standard and delivered the same answer what is the purpose of fisher information? $\endgroup$
    – Damir
    Feb 18, 2015 at 10:02
  • $\begingroup$ never mind. i notice they are the same. essentially i calculated fisher information before with out know the name. Right? $\endgroup$
    – Damir
    Feb 18, 2015 at 10:10
  • 1
    $\begingroup$ The Fisher information is part of the computation for the CRLB but they are not the same. Even when the function $g$ above is the identity function, the CRLB is the reciprocal of the Fisher information. If you only know the formula for estimating a parameter directly instead of estimating a function of the parameter, I suggest you do the same approach above but in terms of $\theta$ by first substituting in $\lambda=\sqrt{\theta}$. $\endgroup$ Feb 18, 2015 at 10:17
  • 1
    $\begingroup$ Answering people's homework for them is not only academically highly inappropriate, it also meddles and corrupts the ability of a university to grade. If you would like to help out, post your answer with a 1 or 2 week delay. That way the student still benefits from the answer, but with minimal risk of inappropriate behaviour. $\endgroup$
    – wolfies
    Feb 18, 2015 at 10:47
  • $\begingroup$ The student clearly likes to learn hence I helped. The ethical burden is on the OP who poses the question, not the answerers, whose answers the OP could decide to ignore. Plus, the question and the answers will be useful for anyone who at any time later has the same concern. $\endgroup$ Feb 18, 2015 at 11:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.