In parametric statistical inference, a statistic $T$ of some variable $X$ (thought of as experimental or observational data) is sufficient for the parameter $\theta$ if is captures the essential information in the data $X$ about the parameter $\theta$ (there are several equivalent definitions for this). It is implicitly understood that the random variable $X$ has a probability distribution $f(x;\theta)$, where the parameter $\theta$ is of course unknown.

Related is the concept of a minimal sufficient statistic, which captures nothing more than the essential.

  1. Is it true that any random variable $X$ as above has a sufficient statistic?
  2. Suppose $X$ has a sufficient statistic $T$. Must it also have a minimal sufficient statistic?

I'd be glad to have either a concise proof or a simple counter-example for each of these two.

  • 1
    $\begingroup$ en.wikipedia.org/wiki/Sufficient_statistic. Wikipedia article on sufficient estimators: A case in which there is no minimal sufficient statistic was shown by Bahadur, 1954. However, under mild conditions, a minimal sufficient statistic does always exist. In particular, in Euclidean space, these conditions always hold if the random variables (associated with $P_{\theta }$) are all discrete or are all continuous. $\endgroup$ – Marc Nov 23 '16 at 15:13
  1. In a degenerate manner your sample $(X_1,...,X_n)$ is always a sufficient statistic from Fisher's information perspective. $T(X)$ is sufficient statistic for $\theta$ iff $\mathcal{I}_{\theta}(T(X))=\mathcal{I}_{\theta}(X_1,...,X_n)$.Thus for every parametric structure you can take the data points themselves as the sufficient statistic.
  2. No. You have a minimial sufficient statistic iff your parametric distribution $f(X;\theta)$ can be factorized into $h(X)g(\theta; T(X))$, then the $T(X)$ is the minimal sufficient statistic or can be reduced to one. Moreover, it is pretty common to fail to satisfy this condition, e.g., Weibull distribution which density function is given by $$ f(x;\theta)=\frac{\theta x^{\theta-1}}{\lambda^\theta}\exp\{-x^\theta/\lambda^\theta\}, x\ge 0, $$
    where $\theta$ is unknown shape parameter, cannot be factorized w.r.t $\theta$.
  • $\begingroup$ Thanks @a-e. Is your answer to (2) not in contradiction to Marc's quote from Wikipedia, "under mild conditions, a minimal sufficient statistic does always exist"? $\endgroup$ – Shlomi A Nov 25 '16 at 21:23
  • 1
    $\begingroup$ Don't know. I guess "mild" it's a relativistic statement. From my humble experience, although the existence of a minimal sufficient statistic is more common in parametric structures, yet a non-existence is not that rare phenomenon. $\endgroup$ – V. Vancak Nov 25 '16 at 21:30
  • $\begingroup$ Alright. Thanks @a-e! As for (1), if I'm interested in non-trivial sufficient statistic, then are there necessary and/or sufficient conditions for their existence? $\endgroup$ – Shlomi A Nov 26 '16 at 11:32
  • 1
    $\begingroup$ If by non-trivial you mean not $(X_1,...,X_n)$ then yes - the factorization theorem is iff statement. I.e., inter alia it provides a sufficient condition for the existence of a non-trivial sufficient statistic. $\endgroup$ – V. Vancak Nov 26 '16 at 11:38
  • $\begingroup$ @V.Vancak Why the sample itself is a statistic? I thought a statistic must be a real-valued function of the sample. $\endgroup$ – Tony B Nov 16 '19 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.