2
$\begingroup$

Are there any grounds for the following claim?

For distributions for which the support in $x$ depends on $\theta$, the CRLB $=0$.

For a quick example (Casella-Berger, Example 7.3.13): Let $X_1,\dots,X_n$ be iid with pdf $f(x\mid\theta) = 1/\theta, 0 < x <\theta$. There, CB used the typical formula for Fisher information $I(\theta)$ to get CRLB $= \theta^2/n$ and subsequently showed that for the unbiased estimator $\hat{\theta} = \frac{n+1}{n}X_{(n)}$, $\mathrm{Var}\hat{\theta} = \frac{1}{n(n+2)}\theta^2$ which is uniformly smaller than the CRLB and thus Cramer-Rao Inequality is violated.

However, my lecturer disagrees with this approach as CB ignored the indicator function $\mathbf{1}_{(0<x<\theta)}$ when taking the derivative of the log-likelihood function. His approach was to note that because the likelihood function is not continuous in $\theta$, it is not differentiable in $\theta$. In situations like these, we should define $I(\theta) = +\infty$ and thus CRLB $= 0$, in which case the Cramer-Rao Inequality was not violated.

I Google'd a bit but did not find any references to validate this alternative approach although it does make some sense (although I can't seem to grasp the significance of having CRLB $= 0$). Has anyone come across something like this before?

$\endgroup$
1
$\begingroup$

The Cramer-Rao lower bound can never be violated if all of the following conditions are verified:

$(i) \quad\Theta$ is an open interval in $\mathbb{R}$

$(ii) \quad\{(x_1,...,x_n)\in \mathbb{X}| \ f_\theta(x_1,...,x_n)>0\} \ \ does \ not \ depend \ on \ \theta$

$(iii) \quad \frac{\partial}{\partial\theta}f_\theta(x_1,...,x_n) \ exists \ for \ each \ \ (x_1,...,x_n) \in \mathbb{X} \ \ and \ \ \theta \in \Theta$

$(iv) \quad \int_{\mathbb{X}} \frac{\partial}{\partial\theta}f_\theta(x_1,...x_n)dx_1...dx_n=0$

Of course, $(ii)$ will not be verified in those distribution families in which the support depends on $\theta$; like the uniform distribution in $(0,\theta)$ you just mentioned above.

Note that in practice the other conditions are straightforward and require no verification, as we deal with smooth functions: note for example that $(iv)$ is equivalent to say that the identity:

$$\int_{\mathbb{X}}f_\theta(x_1,...,x_n)dx_1...dx_n=1$$

can be differentiated under the integral operator

$\endgroup$
0
$\begingroup$

The problem is not with the derivability of the indicator function $\mathbb{I}_{0\le x\le\theta}$ in $f_\theta(x)=\theta^{-1}\mathbb{I}_{0\le x\le\theta}$ at $\theta=x$ since the expectation $$\mathbb{E}^X_\theta\left[\left(\frac{\partial}{\partial\theta}\log\{f_\theta(X)\}\right)^2\right]$$ only requires the integrand to be defined almost everywhere, which is the case if only $X=\theta$ is excluded. The problem is that this expression is no longer the variance of the score function since the expectation of the score is no longer zero, that it no longer defines a lower bound on the variance, that it is no longer connected with the limiting distribution of the MLE, that it is no longer additive in the observations, &tc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.