Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any statistical framework.

Let $\theta\in\Theta\subset \mathbb{R}^n$ be an unknown parameter vector and let $x=f(\theta)\subset\mathbb{R}^m$ be a set of measurements, from which $\theta$ is to be reconstructed. Assume that the absolute error in each measurement is bounded from above by $\epsilon>0$, and otherwise nothing is known about the measurement error. Let $T_{\theta,\epsilon}=f^{-1}(x+B(0,\epsilon))$ (the full preimage, always well-defined). Obviously, $\theta\in T_{\theta,\epsilon}$. The "best possible" reconstruction accuracy is defined as $\delta_{\theta,\epsilon}=\sup_{t\in T_{\theta,\epsilon}} \|t-\theta\|$.

(Informal justification for the definition: if the measurement error is bounded by $\epsilon$, the original parameter vector $\theta$ can only be recovered up to accuracy $\delta_{\theta,\epsilon}$ because any parameter vector $\hat\theta\in T_{\theta,\epsilon}$ could have produced a measurement vector $\hat x \in B(x,\epsilon)$.)

In contrast, Cramer-Rao lower bound depends on some distribution $p(x;\theta)$ of the measurements, and gives a bound on the variance of any unbiased estimator $\hat \theta$ of $\theta$ satisfying some regularity conditions.

Apparently, the two notions are different and in general not comparable.

Still, are there cases in which the Cramer-Rao bound can be thought of as the $\delta_{\theta,\epsilon}$ defined above?


Consider the simple case of a single parameter. Assuming $f'(\theta)\neq 0$, first-order approximation of the reconstruction accuracy as defined above is $$\delta_{\theta,\epsilon}\approx \frac{\epsilon}{f'(\theta)}. \qquad (1) $$ On the other hand, assuming that the measurements are modeled as $$ x = s(\theta) + w$$ where $w$ is Gaussian with zero mean and variance $\sigma^2$, we have (see e.g. Kay's book, section 3.5) $$ CRB\{\theta\} = \frac{\sigma^2}{\biggl(\frac{\partial s}{\partial \theta}\biggr)^2} \qquad (2)$$

Comparing $(1)$ and $(2)$ seems to suggest that in this case the notions are very closely related.

share|cite|improve this question
It's not immediately clear to me that any definition of "unbiased" can make sense in your problem involving "reconstruction accuracy". The Cramer–Rao bound does not apply to biased estimators: there are biased estimators whose mean squared error is less than the Cramer–Rao bound. – Michael Hardy May 23 '12 at 17:06
the modification of the Cramer-Rao bound for biased estimators is simple and standard. – mike May 23 '12 at 22:27
Added the "Example" in the main post. – dima_b May 24 '12 at 9:36

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.