I have some troubles with understanding of this explanation taken from wikipedia:

"An estimator can be unbiased but not consistent. For example, for an iid sample $\{x _1,..., x_n\}$ one can use $T(X) = x_1$ as the estimator of the mean $E[x]$. This estimator is obviously unbiased, and obviously inconsistent."

Why unbiased, and why inconsistent ...

Can someone explain it in more details?

  • $\begingroup$ What do you already know about the definition of each term? $\endgroup$ – cardinal Mar 12 '12 at 22:30
  • 1
    $\begingroup$ I know that unbiased: it means that expected value of parameters obtained from the process is equal to expected value of parameter obtained for the whole population. And that strong consistency means that when number of samples n increases then estimated value almost surely goes to the value of parameter in whole population $\endgroup$ – Darqer Mar 12 '12 at 23:09
  • $\begingroup$ @Darqer:What's unclear? $\endgroup$ – Ashok Mar 13 '12 at 6:00
  • $\begingroup$ I cannot understand how unbiased estimator might be inconsistent. If according to the definition expected value of parameters obtained from the process is equal to expected value of parameter obtained for the whole population how can estimator not converge to parameter in whole population. $\endgroup$ – Darqer Mar 13 '12 at 9:05
  • $\begingroup$ What if I ask Why should the estimator converge to the parameter of the whole population? Let me explain. Suppose $\mu$ is the parameter of the population. Then $E[T(X)]=E[X_1]=\mu$, so $T(X)=X_1$ is an unbiased estimator. But it is not consistent because, for every $n$, $T(X)=X_1$ which doesn't converge to $\mu$ unless $X_1=\mu$ almost surely (i.e. with probability 1). $\endgroup$ – Ashok Mar 13 '12 at 11:52

Suppose your sample was drawn from a distribution with mean $\mu$ and variance $\sigma^2$. Your estimator $\tilde{x}=x_1$ is unbiased as $\mathbb{E}(\tilde{x})=\mathbb{E}(x_1)=\mu$ implies the expected value of the estimator equals the population mean. Your estimator is on the other hand inconsistent, since $\tilde{x}$ is fixed at $x_1$ and will not change with the changing sample size, i.e. will not converge in probability to $\mu$.

Perhaps an easier example would be the following. Let $\beta_n$ be an estimator of the parameter $\beta$. Suppose $\beta_n$ is both unbiased and consistent. Now let $\mu$ be distributed uniformly in $[-10,10]$. Consider the estimator $\alpha_n=\beta_n+\mu$. This estimator will be unbiased since $\mathbb{E}(\mu)=0$ but inconsistent since $\alpha_n\rightarrow^{\mathbb{P}} \beta + \mu$ and $\mu$ is a RV.

  • $\begingroup$ I may ask a trivial Q, but that's what led me to this Q&A here: why is expected value of a known sample still equals to an expected value of the whole population? Intuitively I'd expect expected value of a known sample be equal to itself, e.g. $\mathbb{E}(0) = 0$ $\endgroup$ – Kentzo Oct 9 '18 at 23:59
  • $\begingroup$ @Kentzo Because the sample $\widetilde{x}$ is itself a random variable!! This is very important and is something that I struggled to understand for a long time myself, so I understand your confusion. The best example I could think of is: imagine you are measuring height and are drawing samples (people) at random from the population (humanity). If you know the average height is $\mu = 160$cm, your first sample might be $170$cm, but you still *expect* the average height to be $\mu = 160$, so $\mathbb{E}(\widetilde{x}) = \mu = 160$cm. $\endgroup$ – baibo Jan 1 '20 at 19:12
  • $\begingroup$ Ah, so $\tilde{x} = x_1$ is essentially $\tilde{x} = \tilde{x}$ since $x_1$ can have any value from the population. $\endgroup$ – Kentzo Jan 2 '20 at 20:50

My answer is a bit more informal, but maybe it helps to think more explicitly about the distribution of $x_1$ over repeated samples, with mean $\mu$ and variance, say, $\sigma^2$. $x_1$ is an unbiased estimator for the mean: $\mathrm{E}\left(x_1\right) = \mu$. Roughly speaking, for consistency you would, in addition, need the variance of your estimator to go to zero as the sample size increases. But this doesn't happen here. Even asymptotically, $x_1$ will have a distribution with a non-zero variance, i.e. the distribution doesn't collapse into a single point. Intuitively, no matter how much your sample grows, no additional information is being used to estimate the population mean ($x_1$ still has variance $\sigma^2$ even as $n$ goes to infinity).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.