I'm trying to evaluate a randomized experiment which produces positive or negative results, given an input value. I'm interested in finding all values for which the result is negative, but I get quite a large amount of false negatives.
The errors I get are only false negatives, so as soon as I get a subsequent positive I can be sure that all preceding negative results are false.
I figured I'd just repeat the experiments over and over again to reduce the number of false negatives, leaving me with only real results.
Sadly by testing in a controlled environment I noticed that some false negatives still slip through, meaning that they show up in the final result and are not filtered by a positive result. So I was wondering whether there is something like a certainty measure for this kind of experiment.
For each result I can see the total number of tests, the number of negatives and the number of positives (which is 1 at most since I can be sure of a positive result after the first one). I'd like to know for each negative result how certain it is so that I can simply filter out all results that I cannot be sure about.