# A non-existence example of a sufficient statistic

In parametric statistical inference, a statistic $T$ of some variable $X$ (thought of as experimental or observational data) is sufficient for the parameter $\theta$ if is captures the essential information in the data $X$ about the parameter $\theta$ (there are several equivalent definitions for this). It is implicitly understood that the random variable $X$ has a probability distribution $f(x;\theta)$, where the parameter $\theta$ is of course unknown.

Related is the concept of a minimal sufficient statistic, which captures nothing more than the essential.

1. Is it true that any random variable $X$ as above has a sufficient statistic?
2. Suppose $X$ has a sufficient statistic $T$. Must it also have a minimal sufficient statistic?

I'd be glad to have either a concise proof or a simple counter-example for each of these two.

• en.wikipedia.org/wiki/Sufficient_statistic. Wikipedia article on sufficient estimators: A case in which there is no minimal sufficient statistic was shown by Bahadur, 1954. However, under mild conditions, a minimal sufficient statistic does always exist. In particular, in Euclidean space, these conditions always hold if the random variables (associated with $P_{\theta }$) are all discrete or are all continuous. – Marc Nov 23 '16 at 15:13

1. In a degenerate manner your sample $(X_1,...,X_n)$ is always a sufficient statistic from Fisher's information perspective. $T(X)$ is sufficient statistic for $\theta$ iff $\mathcal{I}_{\theta}(T(X))=\mathcal{I}_{\theta}(X_1,...,X_n)$.Thus for every parametric structure you can take the data points themselves as the sufficient statistic.
2. No. You have a minimial sufficient statistic iff your parametric distribution $f(X;\theta)$ can be factorized into $h(X)g(\theta; T(X))$, then the $T(X)$ is the minimal sufficient statistic or can be reduced to one. Moreover, it is pretty common to fail to satisfy this condition, e.g., Weibull distribution which density function is given by $$f(x;\theta)=\frac{\theta x^{\theta-1}}{\lambda^\theta}\exp\{-x^\theta/\lambda^\theta\}, x\ge 0,$$
where $\theta$ is unknown shape parameter, cannot be factorized w.r.t $\theta$.
• If by non-trivial you mean not $(X_1,...,X_n)$ then yes - the factorization theorem is iff statement. I.e., inter alia it provides a sufficient condition for the existence of a non-trivial sufficient statistic. – V. Vancak Nov 26 '16 at 11:38