# What is the difference between the S.E. of an estimator and the estimator for the S.E.?

I am a bit confused between the standard error of an estimator and the estimator for the standard error. A standard error is the standard deviation of a point estimator. Given i.i.d bernoulli random variables $B_1,B_2,\cdots B_n$, an unbiased estimator for the true proportion of successes is $$P=\frac{\sum\limits_{i=1}^n B_i}{n}.$$ We know that the standard deviation for $P$, or the standard error, is $\sqrt{\frac{p(1-p)}{n}}$ where $p$ is the true success rate of the bernoulli random variables $B$. So $\sqrt{\frac{p(1-p)}{n}}$ is the S.E. of the estimator.

In a hypothesis test, when we derive the null distribution of the estimator, we can mathematically calculate the exact standard error of the estimator because we can assume a $p$, which is often $0.5$, using the formula above. But what if we are not doing a hypothesis testing? Suppose we are now trying to get an estimate for the variance of the estimator for the true proportion. We surveyed 10 samples, each containing $n$ observations of $B$. If we want to estimate the variance of the point estimator, we will have to use the formula $$S^2=\frac{\sum\limits_{i=1}^{10}(p_i-\bar{p})^2}{10}$$ where $p_i$ is the proportion of successes for each sample and $\bar{p}$ is the average proportion of successes across all 10 samples. This formula is an estimator for the variance of the estimator for the true proportion, $p$. So I cannot use an arbitrary $p$, say $p_5$, to estimate the variance of the estimator using the formula $$\left(\sqrt{\frac{p(1-p)}{n}}\right)^2=\frac{p(1-p)}{n},$$ right?

• I ran some simulations using R and it seems like $\frac{p(1-p)}{n}$ is an upward biased estimator for the S.E. – Kun Feb 29 '16 at 22:35

If you are not doing a hypothesis test, an estimator of the standard error you derived for the proportion $p$ is simply $$\sqrt{\frac{\bar x(1 - \bar x)}{n}},$$ which is a statistic that depends only on the observations. Here I used $\bar x$ to denote the sample mean or sample proportion of $1$ outcomes. We can also call it $\hat p = \frac{1}{n}\sum_{i=1}^n B_i$.
• Why is this an estimator? Isn't it just the exact standard error for $p$? An unbiased estimator for the standard error is $S^2=\frac{\sum\limits_{i=1}^{10}(p_i-\bar{p})^2}{10}$ , right? – Kun Feb 29 '16 at 22:09
• An estimator is a statistic that does not depend on any unknown parameters. I didn't say whether the estimator I provided was biased or not; you only asked about the difference in meaning of "standard error of an estimator" and "estimator of standard error." The standard error of an estimator clearly is not itself an estimator, since it is a function of the unknown parameter $p$ (in your case). But if I wanted an estimator of the standard error (of the parameter $p$, one such estimator is the one I wrote in my answer. – heropup Feb 29 '16 at 22:33
• So your answer to my final question is I can use $p_5$ to estimate the variance of the estimator, right? – Kun Feb 29 '16 at 22:42