7
$\begingroup$

I've always heard that the sample mean $\overline{X}$ is "the best estimator" for the population mean $\mu$. But is that always true regardless of the population distribution? is there any proof for that? For example let's suppose for an unknown population, we have three samples, say $X_1$, $X_2$, $X_3$. Based on what I've heard (Not necessarily true) the estimator defined as the following: $$\frac{1}{3}(X_1+X_2+X_3)$$ is always preferable to, for instance: $$\frac{1}{6}(X_1+X_3)+\frac{2}{3}X_2$$ or $$\max(X_1, X_2, X_3)$$ But in what sense is it better? and why?

$\endgroup$
1
  • $\begingroup$ This question might be too broad to answer, but I still like it $\endgroup$
    – Randall
    Commented Aug 23, 2019 at 12:32

2 Answers 2

10
$\begingroup$

It is not true that sample mean is the 'best' choice of estimator of the population mean for any underlying parent distribution. The only thing true regardless of the population distribution is that the sample mean is an unbiased estimator of the population mean, i.e. $E(\overline X)=\mu$.

Now unbiasedness is often not the only criteria considered for choosing an estimator of your unknown quantity of interest. We usually prefer estimators that have smaller variance or smaller mean squared error (MSE) in general, because it is a desirable property to have in an estimator. And it might be the case that $\overline X$ does not attain the minimum variance/MSE among all possible estimators.

Consider a sample $(X_1,X_2,\ldots,X_n)$ drawn from a uniform distribution on $(0,\theta)$. Now $T_1=\overline X$ is an unbiased estimator of the population mean $\theta/2$, but it does not attain the minimum variance among all unbiased estimators of $\theta/2$. It can be shown that the uniformly minimum variance unbiased estimator (UMVUE) of the population mean is instead $T_2=\frac{n+1}{2n}\max(X_1,\ldots,X_n)$. So $T_2$ is the best estimator within the unbiased class where 'best' means 'having the smallest variance'.

$\endgroup$
1
  • $\begingroup$ Thanks for this good answer. $\endgroup$ Commented Aug 23, 2019 at 15:55
4
$\begingroup$

No

No, the sample mean is not always the best estimator. See for instance (besides the example of the mean of a uniform distribution by StubbornAtom) the case of a sample from a Laplace distributed population in the image below (code to compute that image is in this q&a)

showcase that median can be better than mean

In this image you see that the sample median can be a better estimator than the sample mean. Note that the distribution is more concentrated around the true location parameter (in this example this is 0).

Yes

Yes, the sample mean is the best unbiased linear estimator. This is a corrolary of the Gauss Markov theorem, which states that the least squares estimator is the unbiased linear estimator with the lowest variance (best linear unbiased estimator BLUE). The corrolary follows from this theorem because the sample mean is the least squares estimator of a population mean.

That the sample mean is BLUE does not contradict that the best unbiased estimators for the Laplace distribution are the median and for the uniform distribution the maximum. This is because those estimators are not linear estimators.

Linear estimator

A linear estimator is a linear function of the observed $Y$ e.g. in OLS the estimator is a matrix multiplied with the observed values $\hat\theta = M \cdot Y$ where the matrix is $M = (X^TX)^{-1}X^T$)$.

The sample mean $\frac{1}{3} Y_1 + \frac{1}{3} Y_2 + \frac{1}{3} Y_3$ is a linear estimator but the maximum $\text{max}(Y_1,Y_2,Y_3)$ is not.

Note that an estimator like $\frac{1}{6} Y_1 + \frac{1}{6} Y_2 + \frac{2}{3} Y_3$ may actually be possibly best as well. The Gauss Markov theorem is only true for independent errors with equal variance.

For the more general case, if you are considering observations with varying errors and potentially correlated, then you are considering generalized least squares which is also BLUE. For this case you do not consider the simple mean but instead a weighted mean.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .