I am reading a book and there is the following paragraph:

When a survey company decides to call 1,000 people to ask them a few questions, they don’t just pick 1,000 people randomly in a phone booth. They try to ensure that these 1,000 people are representative of the whole population. For example, the US population is com‐ posed of 51.3% female and 48.7% male, so a well-conducted survey in the US would try to maintain this ratio in the sample: 513 female and 487 male. This is called strati‐ fied sampling: the population is divided into homogeneous subgroups called strata, and the right number of instances is sampled from each stratum to guarantee that the test set is representative of the overall population. If they used purely random sam‐ pling, there would be about 12% chance of sampling a skewed test set with either less than 49% female or more than 54% female. Either way, the survey results would be significantly biased.

I am trying to understand where the number ~12% is coming from.

My thought process:

In a population of 1000:

513 are female and 487 are males.

The probability of choosing < 49% females or > 54% female is:

1 - (probability of choosing 49%, or 50%, or 51%, or 52%, or 53%, or 54% females)

Probability of choosing exactly n females for a 100 sized sample from a population of 1000 where 513 of them are females is:

$\frac{{513 \choose n} {487 \choose 100 - n}}{1000 \choose 100} $

So what I did was compute the above for n when it's 49, 50, 51, 52, 53, and 54. Then I summed up the results, using the following code:

from math import comb

total = 1000

males = 487
females = 513

sample = 100

the_sum = 0

for i in range(49, 55):
    the_sum += (comb(females, i) * comb(males, sample - i) / comb(total, sample))

The result was 0.4725453445091902, which is quite not what I had expected. Where did I err?


Binomial distribution code:

total = 1000

the_sum = 0

p = 0.513

for i in range(490, 540):
    the_sum += (comb(total, i) * pow(p, i) * pow(1 - p, total - i))
  • $\begingroup$ You don't know that the group of $1000$ is divided exactly according to the population statistics. That's the entire point. Assume that the probability that a randomly selected person is female is $p=.513$ and use the binomial distribution (or a normal approximation to it) to estimate the probability that your sample is off by some specified level. $\endgroup$
    – lulu
    May 17 at 23:03
  • $\begingroup$ @lulu Thanks! So just to be sure, the probability of choosing exactly n females then becomes ${1000 \choose n} * (0.513^n) * (1 - 0.513)^{1000 - n} $? The result of the code below is 0.8846991409045746, which is just about what I'd expect. Edit: removed code to question for readability. $\endgroup$ May 17 at 23:29
  • 2
    $\begingroup$ Yes, but it is almost certainly easier, computationally, to use the normal approximation. $\endgroup$
    – lulu
    May 17 at 23:36


You must log in to answer this question.

Browse other questions tagged .