# Need to calculate significance from a data set.

I have a set of data that looks like this:

Person 1 [48 total records]
2, 2, 2, 1, 2, 2, 1, 2, 1, 2, 1, 2, 2, 2, 1, 1, 1, 2, 1, 2, 2, 1, 2, 1, 1, 1, 1, 1, 1, 2, 2, 1, 2, 1, 1, 2, 2, 2, 2, 1, 1, 2, 2, 1, 2, 1, 1, 1,
--
Person 2 [56 total records]
1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 1, 2, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 2, 2, 2, 2, 2, 2, 1, 2, 1, 1, 1, 1, 2, 1,
--
Person 3 [18 total records]
1, 2, 1, 2, 1, 1, 1, 1, 2, 2, 1, 2, 2, 2, 1, 1, 1, 1,


A '1' indicates a false answer, and a '2' indicates a correct one. I want to be able to compare records to see if any people in the data set are performing significantly above or below average. I've heard of using z-scores and standard deviation, but I'm not sure if that's the correct approach, or even how I would go about doing the calculation.

I also need to find out the minimum number of records I would need in order to have sufficient confidence in the results.

My math skills are pretty limited, so a simple explanation would be greatly appreciated.

-
It's easy to find scores that are above and below average, but where to draw the boundary between significant and not significant is a personal opinion. Would you mind clarifying what is significant for your purposes? – Matt Calhoun Aug 16 '12 at 2:16
I thought significance was a technical term that indicates whether the product indicates a pattern, rather than just chance. Or am I confusing it with confidence levels? – Jeremy Aug 16 '12 at 5:29

How about a very simple solution that can give you kind of an overview?

1. for each person, calculate the percentage of correct answers (= the number of 2's / total records).
2. group these numbers into groups, e.g. 0-10%, 10-20%, .... Count the numbers in each group. This gives you an overview of how people perform, e.g. 6/10 people belongs to the 80-90% group and 4/10 in the 20-30%.
-
This approach is more reasonable, but many journals in a wide swatch of sciences - particularly biomedicine and healthcare, sociology &c, favor pre-WW2 methods that involve "significance." Stanford statistician Efron in his new book "Large Scale Inference" wrote in a footnote that he no longer refers to patterns as "significant" but rather as "interesting" – alancalvitti Aug 15 '12 at 23:31
Is there a reason why the "significance" approach is a problem? I thought significance was a technical term that indicates whether the product indicates a pattern, rather than just chance. – Jeremy Aug 16 '12 at 5:31