4
$\begingroup$

OK, so I know that the typical standard error of a coin is estimated by $$\sigma_p=\sqrt{ \frac{p*(1-p)}n }$$ where $p$ is the estimated probability and and $n$ is the number of samples. This seems reasonable at high $n$ and $p \sim 0.5$; however, it seems unreasonable if I have $p = 1$ and $n = 20$, $\sigma_p = 0$.

Is there a better formula for standard error when $ p \sim 0$ or $p \sim 1$ and $n$ is low?

Note: this is a real-world problem and increasing $n$ is non-trivial.

Thanks!

$\endgroup$
6
  • $\begingroup$ If p is one or zero then there isn't anything you can do since you don't have any data on other events. If n is small then your distribution will not be approximately normal. I would reccomend just looking at a binomial distribution instead. I think it will be more useful for what you're trying to do. $\endgroup$
    – Zaros
    Dec 12, 2016 at 22:27
  • $\begingroup$ Although I cannot offer any useful insight to this problem, I agree with the above comment. If you already know p~0, or p~1, then why even estimate the standard error anyways? $\endgroup$
    – Brandon
    Dec 12, 2016 at 22:41
  • 1
    $\begingroup$ Thanks guys, basically, what I need is an estimate of the true probability with 95% CI. Perhaps I am approaching this wrong. Is there a more appropriate way to determine the 95% confidence interval of my true probability when my estimate probability is near the extremes? $\endgroup$
    – ipetrik
    Dec 12, 2016 at 22:46
  • 1
    $\begingroup$ I think I answered my question - I guess I need a more exact method of computing a binomial confidence interval - Clopper-Pearson? $\endgroup$
    – ipetrik
    Dec 12, 2016 at 23:03
  • 2
    $\begingroup$ The Agresti binomial 95% CI is almost as good as Clopper-Pearson, and considerably simpler. For $X$ successes in $n$ trials, use $\tilde n = n + 4,\, \tilde p = (X + 2)/(n+4).$ Then the Agresti CI is $\tilde p \pm 1.96 \sqrt{\tilde p(1- \tilde p)/\tilde n}.$ For levels other than 95% C-P can be a lot better. $\endgroup$
    – BruceET
    Dec 13, 2016 at 1:07

1 Answer 1

0
$\begingroup$

If $p = P(S) = 1,$ then $X \sim Binom(n, p),$ has $X \equiv n$ and $Var(X) = 0 = \sqrt{p(0)/n}$ so the formula for the variance works fine.

In public opinion polls, the margin of sampling error is often given as $\pm \sqrt{1/n},$ which comes from the largest possible variance at $p = 1/2.$ Then the margin of error for a 95% confidence interval (using the normal approximation for large $n$) is about $$\pm 1.96\sqrt{p(1=p)/n} = \pm 1.96\sqrt{1/4n} \approx \pm \sqrt{1/n}.$$

Thus, $n = 2500$ subjects give a margin of sampling error $\pm 2$%, and $n = 1100$ subjects give a margin of sampling error of about $\pm 3$%.

In one sense, this vastly over-estimates the margin of error for cases in which $p$ is near 0 or 1. But pollsters understand that non-sampling errors (lack of response, unwillingness to give honest responses, sampled population differing from target population, and so on) can be especially serious for such extreme values of $p.$ So they use $\pm \sqrt{1/n}$ anyway, hoping to cover all contingencies.

Perhaps you have some intuition about practical difficulties in sampling when $p$ is far from 1/2 that is responsible for your doubts about the variance formula. But as an exact mathematical statement about sampling error only, the formula is correct.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .