# In the definition of a complete statistic, what is the expectation taken over?

I have read lots of answers here attempting (and in my case, failing) to convey the idea of a "complete statistic."

Even the idea of $$E_{\theta}[g(T(x))] = 0$$ is hard to parse. What is the expectation taken over? Different data? Different values of $$T$$? Different functions? Different values of a given function? Different values of $$\theta$$?

The binomial example would suggest it is values of $$g(T(x))$$ and their probabilities that are being summed over. Is that correct? If so, is it still a function of $$\theta$$?

I have Casella and am scrutinizing it along with lecture notes from a stats class online. Any pointers to good references or explanations are welcome.

EDIT to add detail. So I'm getting the impression that the space is some sort of space of functions that includes every distribution under consideration, I think. So what else is in that space? What isn't? All functions? All continuous functions? What are we trying to accomplish in that space? What's the inner product?

EDIT to add more detail. I created an example.

Suppose I have two dice: one fair, and the other has even numbers twice as likely to come up as odd numbers. So $$\Theta$$ consists of two possible values, call them $$a$$ and $$b$$.

Suppose $$T(X)=1$$ if $$X$$ is even, and $$0$$ otherwise.

Suppose $$U(i) = i$$ for $$i \leq 4, U(5)=3$$ and $$U(6) = 4$$.

I have computed that both $$T(X)$$ and $$U(X)$$ are sufficient statistics, and $$T(X)$$ is a minimal sufficient statistic.

So in addition to the desired general explanation, I would also like answered a more specific question, that I would like to see expressly calculated, is, "Are either $$T$$ or $$U$$ complete?"

• Has it been clarified what the meaning of $\Bbb E_\theta[g(T(X))]$ is? $X$ is a random variable with law $\Bbb P_\theta$; this is a probability distribution which depends on $\theta$, an unknown parameter. $g(T(X))$ is a function of this random variable, we take its expectation with $\theta = \theta_0$ and call it $\Bbb E_{\theta_0}[g(T(X))]$, or $\Bbb E[g(T(X))|\theta_0]$. For example, in the binomial example, the parameter is $p$ and $\Bbb P_p \sim \mathrm{Ber}(p)$. Feel free to ask if unclear. Jul 16, 2021 at 23:47

### Setup

You have a parameter space $$\Theta$$ consisting of parameters $$\theta$$ for probability distributions $$P_\theta$$. (For example, $$\Theta = [0,1]$$ is a parameter space for the family of $$\text{Bernoulli}(\theta)$$ distributions.)

For a fixed $$\theta$$, the data $$X$$ is drawn from $$P_\theta$$. For each value of $$\theta$$ you will have a different data-generating distribution $$P_\theta$$, so the behavior of $$X$$ will change. This distribution $$P_\theta$$ is the only source of randomness in the model. It may be helpful to think of $$\theta$$ as a fixed value for now.

A statistic $$T(X)$$ is a measurable function applied to the data $$X$$. It is a random variable because it depends on $$X$$ which itself is a random variable that follows the distribution $$P_\theta$$. The distribution of $$T(X)$$ depends on the value of $$\theta$$ because the distribution of $$X$$ depends on the value of $$\theta$$.

The $$g$$ in the definition of completeness is a measurable function. $$g(T(X))$$ is a random variable as well because it is a function of $$X$$, and $$E_\theta[g(T(X))]$$ is the expectation of this random variable taken with respect to the randomness in $$X$$, i.e. the distribution $$P_\theta$$. This is why there is a subscript $$\theta$$ in the expectation.

So, for fixed $$g$$ and $$\theta$$, the quantity $$E_\theta[g(T(X))]$$ is the expectation of a function of $$X$$. For different values of $$\theta$$, this quantity may differ.

### Definition of completeness

Suppose $$g$$ is a measurable function such that

$$E_\theta[g(T(X))] = 0 \text{ for every \theta \in \Theta}\tag{*}$$

Remember, changing $$\theta$$ means changing the distribution of $$X$$. But somehow, applying the function $$g \circ T$$ on $$X$$ results in a zero-mean random variable $$g(T(X))$$ no matter what the distribution of $$X$$ is. One simple way this could happen is if $$\text{g(T(X)) is zero with probability 1 (i.e. P_\theta(g(T(X))=0)=1) for all \theta \in \Theta} \tag{**}$$ (e.g. if $$g(z) = 0$$ for any $$z$$ then $$E_\theta[g(T(X))]=E_\theta=0$$). But this might not be true, it may be that $$g(T(X))$$ are nontrivial random variables that just all happen to have zero mean. If ($$**$$) is the only way the condition ($$*$$) can happen, then we call $$T(X)$$ complete.

If the above holds for bounded measurable functions $$g$$, we call $$T(X)$$ boundedly complete.

### Motivation

Completeness, when combined with sufficiency, is a strong assumption on a statistic that leads to several important results: see the Lehmann-Scheffé Theorem, Basu's Theorem, and Bahadur's Theorem discussed here. The answers to this question use these theorems to give more explanation about why completeness is a useful concept.

• I followed a lot of that, thanks. But if X is a lot of data, isn't T(X) one number for the data set, such as a sample mean? But you talk about g(T(X)) in the plural. So if T is a vector of functions of the data, does g(T(X)) also make a vector? So g(T(X))=0 means it equals the zero vector? Jul 16, 2021 at 5:46
• @RobertTheTutor My reference to $g(T(X))$ in the plural is referring to the fact that for each $\theta$ the random variable $g(T(X))$ is different. I think it suffices to consider functions $g$ whose output is in $\mathbb{R}$; the cases where $g$ outputs a vector can be reduced to the univariate case. Jul 16, 2021 at 7:43

I feel like this is not going to be a popular answer among statisticians, but this is how I think about the whole story. Here goes, hopefully it is of some help.

Let me explain the standard fuzz in a few lines: in the background, what statisticians have is a parametric family of probability distributions $$\{\Bbb P_\theta : \theta \in \Theta\}$$. You don't know the law of the random variables at hand; they can be anything depending on what the parameter $$\theta$$ is. Goal of a statistician is to guess what the distribution is from the experiments. Remember throughout that $$\theta$$ is unknown, varying in the parameter-space $$\Theta$$.

Let's say as a simplifying assumption that the range of the probability distribution is all of $$\Bbb R$$, and let's also assume these are absolutely continuous. If one does $$n$$ experiments, the results are the random variables $$X_1, \cdots, X_n \sim \Bbb P_\theta$$. The join vector $$(X_1, \cdots, X_n)$$ is something in the Euclidean space $$\Bbb R^n$$. This is the space of all possible results of $$n$$ experiments. A statistic is pretty much just a measurable function $$T : \Bbb R^n \to \Bbb R$$. I find it convenient to think of it as a partition of the space of results of $$n$$ experiments, $$\Bbb R^n$$, into a continuum of subsets $$\{T^{-1}(r) : r \in \Bbb R\}$$. A foliation in the geological sense if you wish (to a differential geometer, a foliation means something a bit more specific).

The varying probability distributions $$\Bbb P_\theta$$ can be thought of as a heat-distribution on $$\Bbb R^n$$. The law of $$(X_1, \cdots, X_n)$$ is $$\Bbb P_\theta^{\otimes n}$$, of course, so one can imagine that an evolution of a heat distribution on $$\Bbb R^n$$ with time $$\theta$$; at time $$\theta = \theta_0$$, some places are "hotter", aka there's more probability for the results of the $$n$$ experiments to land there, as opposed to other places which are "colder", which are sort of outlier regions for the result of the experiments. The heat diagram at time $$\theta = \theta_0$$ is the same as what simulation of the results of the $$n$$ experiments for the parameter $$\theta = \theta_0$$ would give.

On thing to note is that the $$T$$ associates to each leaf $$T^{-1}(r)$$ the number $$r$$. So our decomposition has moreover a value associated to each leaf; $$T(X_1, \cdots, X_n)$$ is the random variable which picks out a leaf at random according to the law of $$\Bbb P_\theta$$ (by picking a point at random from $$\Bbb R^n$$ according to $$\Bbb P^{\otimes n}_\theta$$, and then picking the unique leaf which passes through it, in case this is still unclear), and spits out the value of that leaf. To a mathematician, this is a probability distribution on a "space of leaves", $$\Bbb R^n/T$$ given by collapsing each leaf $$T^{-1}(r)$$ down to a point. Once again, $$\theta$$ is varying, so it is an evolution happening on the leaf space as well. The expected value $$\Bbb E_\theta[g(T(X_1, \cdots, X_n))]$$ is the expected value of the leaf, with respect to the probability-distribution on the leaf space induced from $$\Bbb P_\theta$$. So you can consider it to measure the "expected leaf".

Here's an immediate application of these intuitive ideas: What is a sufficient statistic? Recall the partition $$\Bbb R^n = \bigsqcup_{r \in \Bbb R} T^{-1}(r)$$ Sufficiency demands that $$T(X_1, \cdots, X_n)$$ is independent of $$\theta$$; this is the same thing as saying that the heat evolution on $$\Bbb R^n$$ does not evolve on $$T^{-1}(r)$$'s for $$r \in \Bbb R$$. On the plaques or leaves of the "foliation", you cannot detect any change in the heat distribution as $$\theta$$ varies; indeed, the way heat evolves is if the leaves $$\{T^{-1}(r) : r \in \Bbb R\}$$ come closer, or move further. This "foliation" is the contour plot of the heat evolution! Another way to state it is $$\Bbb P_\theta$$ is completely determined by the induced distribution on the leaf space $$\Bbb R^n/T$$; there is no information "along the leaves", only "transverse to the leaves".

It's trickier to understand what completeness means. If $$g : \Bbb R \to \Bbb R$$ is another measurable function, then $$g \circ T : \Bbb R^n \to \Bbb R$$ is a new statistic, obtained from scrambling the values of the statistic $$T$$ by $$g$$ (this scrambling is not too drastic, in other words it's a measurable scrambling). The new decomposition is

$$\Bbb R^n = \bigsqcup_{r \in \Bbb R} T^{-1}(g^{-1}(r))$$

This is obtained from the old decomposition by simply collecting some of the leaves of the older foliation into a fat leaf of the new foliation. In other words, $$T^{-1}(g^{-1}(r))$$ is a union of the $$T^{-1}(s)$$, $$s \in g^{-1}(r)$$. Alternatively the leaf space $$\Bbb R^n/g \circ T$$ is a further quotient of $$\Bbb R^n/T$$.

Completeness is the same as saying that unless $$g \circ T \equiv c$$ is a constant a.e., i.e., the new scrambled foliation is the trivial one (if $$T \equiv \mathrm{const}.$$, what's the foliation corresponding to $$T$$ of $$\Bbb R^n$$?), one can find $$\theta_0 \in \Theta$$ such that expected leaf at $$\theta = \theta_0$$ has some value other than $$c$$. The expected leaf cannot have the same value throughout the heat-evolution; the value (i.e., the statistic $$T$$ or $$g \circ T$$ after scrambling) is supposed to capture something about the change of the heat-distribution, in expectation. That's all.

Here's another way to write down the definition of completeness:

Let $$X$$ be a random variable from a family parametrised by $$\theta \in \Omega$$, where $$\Omega$$ is some parameter space. We say that the random variable $$T = T(X)$$ is a complete statistic for $$\theta$$ if for any measurable function $$h$$ we have $$E(g(T)) \quad \forall \theta \in \Omega \implies P(g(T) = 0) = 1$$

In particular, this expectation is taken with respect to the distribution of $$T$$. The reason $$\theta$$ appears as a subscript in your definition is because $$T$$ depends on $$X$$, whose distribution in turn depends on the value of $$\theta$$.