The standard error is defined: $$ SD(\bar{X}) = \frac{\sigma}{\sqrt{n}} $$ where $\sigma$ is the standard deviation of the population and $n$ is the sample size. In the book I'm reading, it seems to be saying the $SD(\bar{X})$ should be as small as possible, but shouldn't it rather be as close as possible to the actual standard deviation $\sigma$, rather than approaching zero? How have I misunderstood the what the standard error tells us?
5
-
$\begingroup$ Answer to the first question: s.d. represents an estimation of the variance. So, the answer to the second question: s.d. tells us almost the same as variance (measure of the data variety). $\endgroup$– openspaceDec 7, 2022 at 11:34
-
$\begingroup$ The standard deviation is the square root of the variance, no? But in any case I am asking about the standard error, not the standard deviation. $\endgroup$– Dan ÖzDec 7, 2022 at 11:54
-
$\begingroup$ The sample mean (of n values of a variable X) is a different random variable to X. It has the same mean as X but has a smaller standard deviation, as you can see. If n is reasonably large, the sample mean random variable starts to approximate a normal random variable, even if X is not itself normal. This is exploited in testing for example, which uses samples of data values rather than individual X values. $\endgroup$– PaulDec 7, 2022 at 11:55
-
$\begingroup$ You have written down the standard deviation for a sample, not the standard error. The standard error also has a multiplicative factor (1.96, $t_{n-1}$, ) $\endgroup$– PaulDec 7, 2022 at 11:58
-
$\begingroup$ Colloquially if your data has no error there should be no deviation. You'd expect data without errors to also have no deviation so any definition of error should include that characteristic. $\endgroup$– CyclotomicFieldDec 7, 2022 at 12:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There are two different quantities:
- standard error: variance of the estimator that serves to estimate the mean. $V(\hat{X}) = nV(X)/n^2=V(X)/n$ therefore $\sqrt{V(\hat{X})} = \sigma/\sqrt{n}$ as you have corectly indicated. This number tends to 0 as $n$ grows to infinity (this is the law of large numbers)
- population variance: the variance of the samples that can be estimated through a different estimator, say, $\hat{V}$. This random variable, $\hat{V}$ is what you would like to approximate the actual variance. Alternatively, $\hat{D}$ could represent an estimator for the standard deviation, and you would like it to approximate the standard deviation, not zero, as you correctly pointed out.
An estimator is a function of the random variables, so it is also a random variable. The estimator has a mean and a variance. In general, you want the mean to be close to the value you want to estimate, and the variance to be as close as possible to zero.
Examples of estimators pointed out above
$$\hat{X}= \sum X_i/n$$
$$\hat{V} = \sum ( X_i-\hat{X})^2/n $$ or $$\hat{V} = \sum ( X_i-\hat{X})^2/(n-1) $$
$$\hat{D} = \sqrt{\sum ( X_i-\hat{X})^2/n} $$ or $$\hat{D} = \sqrt{ \sum ( X_i-\hat{X})^2/(n-1) } $$
