The Rao-Blackwell theorem states that conditioning an unbiased estimator of some parameter $\theta$ on a sufficient statistic can only reduce the variance of the estimator. However, couldn't we say that more generally, conditioning an unbiased estimator on any random variable, in fact on any sigma algebra at all, can only reduce the variance?

Let $\Sigma$ be some sigma algebra, $T$ an unbiased estimator for $\theta$, and $T'=E(T|\Sigma)$:

$$ \begin{align} E((T'-\theta)^2) &= E((E(T|\Sigma) - \theta)^2) \\ &= E(E(T-\theta | \Sigma)^2) \\ &\leq E(E((T-\theta)^2 | \Sigma)) \quad\text{(convexity)}\\ &=E((T-\theta)^2) \end{align} $$

  • $\begingroup$ but will it remain a statistic? $\endgroup$ – spaceisdarkgreen May 7 '17 at 16:19
  • $\begingroup$ @spaceisdarkgreen Won't it? Obviously I'm taking $\Sigma$ to be a sub-sigma algebra of the ambient sigma algebra. $\endgroup$ – Jack M May 7 '17 at 16:21
  • 1
    $\begingroup$ I don't think so. Generally the conditional expectation of a statistic will depend on the parameter unless you're conditioning with respect to a sufficient statistic. (This is just the definition of sufficiency when you think about it). $\endgroup$ – spaceisdarkgreen May 7 '17 at 16:23
  • $\begingroup$ @spaceisdarkgreen Okay, I see the problem - if $\Sigma$ is the trivial tribe, for example, the estimator for $\theta$ you end up with is just $\theta$ itself, not very useful. In fact $T'$ is indeed "a statistic" in some sense - it's definitely $\sigma(X)$ measurable, where $X$ is the observation - but the real problem is that $T'$ doesn't even refer to a single random variable, you would get a different random variable for each value of $\theta$. $\endgroup$ – Jack M May 7 '17 at 16:36
  • 2
    $\begingroup$ Yep. Statistic means it's only a function of the data and not the parameter. The conditional expectation is useless as an estimator if it depends on the parameter. (Alternatively if we're being Bayesian and our space includes the parameter as a RV, it won't be $X$-measurable.) $\endgroup$ – spaceisdarkgreen May 7 '17 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.