# Statistics question about sample size and misleading information

In my Stats class, I was told n>=20 is the cutoff for useful information. At work, our system is programmed to classify anything under n>=30 as misleading. (We use a lot of customer surveys.)

Which of these, if either, is correct?

• These sorts of cutoffs are more "guidelines" than actual rules. – Alex Becker Aug 19 '13 at 18:06
• Yes, often 20 is sufficient to derive significant results. More important than the data-sizes are the p-values: en.wikipedia.org/wiki/P-value. – badroit Aug 19 '13 at 18:07

## 2 Answers

It doesn't make sense to consider the usefulness of a sample based on sample size.

For example, let's say I want to see if some function is generating random integers (lets keep it $<2^{32}$) or is generating the same number each time. Let's say I get two outputs from that function and I get 42 twice. Now I can say with high confidence (about 1 in $2^{32}$ probability of being wrong) from a sample size of only 2 that the function is not a random integer generator. I've drawn a useful conclusion with high confidence over 2 data points.

Thus, more important than the data-sizes are the p-values, which tells you the probability of the null hypothesis (in this case, the function returns the same number each time) being true. In the case above, $p = \frac{1}{2^{32}}$. By convention, if $p < 0.05$, the result is considered significant.

If you are only doing one survey, then you can get away with using smaller sample sizes to establish statistically significant results and/or accurate representation of population views. But if you are using lots of surveys at work, then the problem with using the same sample size as what you would use for just one survey is that you could get unrepresentative results in some of your surveys just by chance, because you have conducted so many surveys. So if you want to be pretty sure that ALL (or all but a very few) of your surveys are in fact accurate representations of the population, then you need to be more lenient about how wide you make your confidence regions for each survey, to guarantee that it is likely that all (or all but a few) of the surveys give results that are actually accurate within the stated confidence regions. And if you don't want to lose many of your statistically significant findings as a result of having wider confidence regions, then you need a larger sample size so that the confidence regions start off smaller which counters the effect when you make them larger to account for so many survey results being used.