# Number of Data Samples Needed for 80% Confidence Level for Flight Test?

Simply trying to determine the minimum number of times to repeat a test that will give us an 80% confidence level that Mode A is tracking better than Mode B.

Each test will involve an aircraft sensor tracking a moving target in a two different modes (e.g., A&B), executed one right after the other. The flight data will be qualitatively examined post-test to determine the result of either "yes" the sensor is tracking better in Mode A than Mode B or "No" Mode A is not tracking better.

We're trying to get an estimate of the minimum number of times to repeat the test since flight testing is very expensive and extensive test planning is required to expend resources wisely.

I'm rusty on the statistics so not sure if we need to assume a confidence interval or anything else?

• The problem is that if Mode B has a 51% chance of being better than Mode A then it will take substantially more runs to prove that than say if Mode B is better 90% of the time. – Jared Jun 3 '15 at 3:44
• You should Google "Classical Credibility" – MPW Jun 3 '15 at 4:16
• Here are two questions I would ask yourself: 1) How many test cases does it take to prove that Mode A is better when Mode A is better 100% of the time? and 2) How many test cases does it to take prove that Mode A is better when Mode A is better 75% of the time? – Jared Jun 3 '15 at 5:39