What do I need in order to draw conclusions from this data? I have three techniques, called A, B and C. Each can be used independently when trying to perform four related tasks (Tasks 1, 2, 3 and 4). I have run lots of tests, and tried all combinations of each technique being on or off. My results look something like this. Let's say that higher numbers are better.
$$
\begin{array}{l|r|r|r|r|r|r|r|r|r|}
\mbox{Technique $A$} & - & - & - & - & X & X & X & X \\
\mbox{Technique $B$} & - & - & X & X & - & - & X & X \\
\mbox{Technique $C$} & - & X & - & X & - & X & - & X \\ \hline
\mbox{Task $1$} & 433 & 277 & 911 & 492 & 686 & 4211 & 3775 & {\bf 9732}\\
\mbox{Task $2$} & 149 & 1063 & 5562 & {\bf 6035} & 3 & 58 & 1391 & 1708\\
\mbox{Task $3$} & 220 & 1278 & 7014 & {\bf 7018} & 10 & 97 & 2083 & 4452\\
\mbox{Task $4$} & 218 & 1255 & 6142 & {\bf 8656} & 1 & 73 & 1087 & 2056\\
\end{array}
$$
Looking at the numbers, it seems that $B+C$ is good for Tasks 2,3 and 4, and that $A$ on its own is best for Task 1. But I want to say a bit more. I'd like to be quantitative if I can. My question is: can I deduce anything quantitative from this data? Or do I really need some measure of the variance of the observations? That is, I suspect the numbers might be different if I ran all the tests again.
 A: Since you are exploring a categorical variable (success-failure), the appropriate test is a chi-square test. You have to perform separate analyses for the four tasks. For each task, build a $2\times8$ contingency table where the 8 columns represent the different method (single or combined techniques) and the 2 rows represent successes and failures. Then fill each cell with the number (not percentage) of successes and failures, respectively (the latter are clearly given by 100000 - number of successes), for each method. Lastly, run the chi-square test for this $2\times8$  table using one of the several statistical softwares or online tools. Repeat this analysis for each task.
For tasks where you get a significant p value, you can then use some additional procedures for pair-wise comparisons of proportions. These methods, (e.g., the Marascuilo procedure, the "partitioned chi square" approach, the Holm–Bonferroni method, and so on) allow multiple comparisons for proportions when we want to test which particular proportions are different from each, and can be used when  the null hypothesis has been rejected an overall chi-square test. This further analysis could give you additional information on what is the best method.
