hypothesis testing: A better fly killer

Say I have to prove that a new fly killer is better than an existing one and I set up an experiment where I put both fly killers in a room of live flies and empty and count how many flies are killed by each fly killer every 10 minutes.

The fly killers don't interfere; the past number of flies killed does not affect the number of flies available for killing.

If I have counts of flies killed by each fly killer every 10 minutes, I can work out mean fly kills for each product and compare them, but how do I determine that I have enough samples to work out if any difference is significant?

What if I calculate standard deviations of the means for each fly killer, Can I use these to make a better statement of the validity of any statement that one is better than another?

I should just state that my statistical knowledge is not great, I am an engineer that found the term "hypothesis testing" just today when doing my own search.

Thanks.

-

First, read up on the basics of hypothesis testing, i.e., Null vs Alternative Hypotheses, the role of the test statistic and rejection region, Type I vs Type II error. After that, you will want to model the number of flies caught every 10 minutes as a Poisson random variable with mean rate $\lambda$ flies per 10 minutes. Then, I think this post will answer your questions; http://stats.stackexchange.com/questions/9561/checking-if-two-poisson-samples-have-the-same-mean

-
I am in the midst of learning about hypothesis testing at the moment. Thanks for the link to the other Q&A - I'll follow that lead too. Thanks. – Paddy3118 Nov 14 '13 at 21:58

What I have so far.

Assume normal distribution of flies killed by either flykiller. Call the flykillers X and Y I can calculate the mean and sd of the kill rates of each flykiller: Xmean, Xsd, Ymean, Ysd

From https://en.wikipedia.org/wiki/Cumulative_distribution_function it seems that we can use the cdf to calculate the probability "that a real-valued random variable X with a given probability distribution will be found at a value less than or equal to x"

So for any kill rate k shouldn't I be able to calculate the probability of X >= Y as:

(1 - cdf(k, Xmean, Xsd)) * cdf(k, Ymean, Xsd)

I.e. the probability that X > k times the probability that Y < k = probability that X >= Y

If we calculate the above for all k wouldn't we be able to get the maximum probability that X >= Y and as long as that probability is below a threshold, say 5% could I then go on to say that Y > X with 95% confidence?

-