Finding the minimum sample size of an opinion poll

I have encountered the following problem in a textbook,:

Assume that 28 % of voters favored party A at some point. A later opinion poll gave a result of 30 % of voters favoring party A. What is the minimum sample size allowing us to discard the null hypothesis of no change in voter preference, using a significance level of 5%?

I started out by treating the poll as a binomial test, which can be approximated by a normal distribution for large sample sizes.

Let $X$ be a stochastic variable describing the number of voters expressing preference for party A in the poll. For a binomial test, the expected value of $X$ is $E(X)= n \cdot p$ and $Var(X) = np(1-p)$

Let $Y$ be a stochastic variable describing the proportion of voters expressing preference for party A in the poll. Then, we have

$$Y = X/n = (1/n)\cdot X$$

For a variable $Y = aX$, we have

$$Y = aX \to Var(Y) = {a^2}Var(X)$$

Then,

$$Var(Y) = (1/n)^2 \cdot Var(X) = (1/n^2) \cdot np(1-p) = (p/n)(1-p)$$

Then, the standard deviation of Y, the proportion of voters expressing their preference for party A will be

$${\sigma _Y} = \sqrt {\frac{1}{{{n^2}}}np(1 - p)} = \sqrt {\frac{p}{n}(1 - p)}$$

A right-sided significance test with a significance level of 5% gives

$$P(Z \geqslant z) < 0.05 \to P(Z \leqslant z) > 0.95$$

$$P(Z \leqslant z) > 0.95 \to z > 1.65$$

And the conversion to standard normal distribution gives

$$Z = \frac{{y - \mu }}{{{\sigma _y}}} = \frac{{y - \mu }}{{\sqrt {\frac{p}{n}(1 - p)} }}$$

Then we get the inequality

$$z > 1.65 \to \frac{{y - {\mu _Y}}}{{\sqrt {\frac{p}{n}(1 - p)} }} > 1.65$$

Solving the inequality for $n$:

$$y - {\mu _Y} > 1.65 \cdot \sqrt {\frac{p}{n}(1 - p)}$$

$${\left( {y - {\mu _Y}} \right)^2} > {1.65^2}\frac{p}{n}(1 - p)$$

$$n{\left( {y - {\mu _Y}} \right)^2} > {1.65^2} \cdot p(1 - p)$$

$$n > \frac{{{{1.65}^2} \cdot p(1 - p)}}{{{{\left( {y - {\mu _Y}} \right)}^2}}}$$

Substituting $p = 0.28$, $y =0.3$ and ${{\mu _Y}} = 0.28$

$$n > \left( {\frac{{{{1.65}^2} \cdot 0.28(1 - 0.28)}}{{{{\left( {0.3 - 0.28} \right)}^2}}} = 1372.14} \right)$$

This way, I ended up with a minimum sample size of $n>= 1373$.

Is my thinking correct?

• there's a StackExchnageSite for statistics too, try there? – RE60K Mar 15 '15 at 17:11
• Good idea. Maybe a moderator can move it? – jarlemag Mar 15 '15 at 17:23

Your method is OK if you take the former value 28% to be exactly correct. But it was probably the result of a previous poll with its own margin of error. Thus, if you really what to know whether there has been "an increase from before" rather than "an increase from 28%", you need to compare results of the two polls. A standard test for judging the "difference between two population proportions" is illustrated as follows.

Suppose the former poll interviewed $n_1 = 900$ subjects to get $\hat{p_1} = .28$, that means they had $X_1 = 252$ respondents in favor of Party A. Also suppose you interview $n_2 = 1400$ subjects to get $\hat{p_2} = .30$ based on $X_2 = 420$ in favor of Party A. We want to test the null hypothesis that $p_2 = p_1$ against the alternative that $p_2 > p_1$. Under the null hypothesis that the two population proportions are equal, the estimate of the common proportion is $\hat p = (X_1 + X_2)/(n_1 + n_2)$.

The approximately standard normal z-statistic for this test is $$z = [(\hat{p_1} - \hat{p_1}) - (p_1 - p_1)]/SD(\hat{p_2} - \hat{p_1}),$$ where the square of the denominator is estimated by $\hat{\sigma^2} = \hat p(1-\hat p)(1/n_1 + 1/n_2).$

For the values I'm using as an example, we have $\hat p = 0.2922$, $\hat\sigma = 0.0196$, and then $z = (.02 - 0)/ 0.0196=1.02.$ Because $z < 1.645$ we cannot reject the null hypothesis that the two polls agree.

You might protest that I've cheated by imagining a small sample size for the first poll, but the point is that the $n_1$ really matters. You can try it for the assumption that $n_1 = n_2 = 1400$ to see what happens.

Caution: Of course, all of this takes account only of sampling error under ideal conditions. Sampling bias is not taken into account.

Addendum 1: A very rough rule of thumb linking sample sizes and margins of error is that the margin of error plus or minus is is about $1/\sqrt{n}$. It works best if the population proportion is about 1/2, with somewhat smaller margins of error for p's that differ from 1/2. So the margin of error for a poll with $n = 2500$ is about $\pm 2\%$. That is the basis of the "margins of error" you see printed in newspapers and mentioned on TV. For all practical purposes you have to regard a difference of 2% or 3% between two polls of typical size in campaigns as being inconsequential. If several polls in sequence all show increases of this size, then some optimism for Party A supporters is warranted.

Addendum 2: If you are looking at the difference between two major candidates in the same poll, then the margin of error for the difference is essentially double the margin of error for each individual candidate. If Candidate X gets 45% and Candidate Y gets 48% in a poll with $n = 2500,$ that is a "statistical dead heat."

• Good point. I have changed the wording slightly from the original, as I had to translate it. The 28 % figure was originally stated as being an election result. Of course, the issue remains as we have to assume that the election result is identical with the expectation value for the polling result, which I don't think is neccesarily realistic. Thanks for the input! – jarlemag Mar 16 '15 at 15:29