# Evaluating randomized experiment with false negatives

I'm trying to evaluate a randomized experiment which produces positive or negative results, given an input value. I'm interested in finding all values for which the result is negative, but I get quite a large amount of false negatives.

The errors I get are only false negatives, so as soon as I get a subsequent positive I can be sure that all preceding negative results are false.

I figured I'd just repeat the experiments over and over again to reduce the number of false negatives, leaving me with only real results.

Sadly by testing in a controlled environment I noticed that some false negatives still slip through, meaning that they show up in the final result and are not filtered by a positive result. So I was wondering whether there is something like a certainty measure for this kind of experiment.

For each result I can see the total number of tests, the number of negatives and the number of positives (which is 1 at most since I can be sure of a positive result after the first one). I'd like to know for each negative result how certain it is so that I can simply filter out all results that I cannot be sure about.

-
If a given input value produced only negative results, it is difficult to imagine a reasoning, mathematical or otherwise, which would lead to the conclusion that these are all false negative. –  Did Jan 25 '12 at 18:54
My first guess was that I could use the number of negative results (which due to the randomness is not all equal) to estimate how likely it is that a given value is produces a real negative result. That is I trust a 4 times negative much more than a 2 time negative, in that the chance of getting 4 false negatives would be much lower. Does it make sense to think of it this way? I assume that false negatives are randomly distributed. –  cdecker Jan 25 '12 at 22:30