# Pearson's Chi Squared / Cochran–Mantel–Haenszel test analog to N-way ANOVA

A test was given to two sets of students, CONTROL and EXPERIMENT, that had question A and question B. I want to know if students who got question A right were more likely to get question B right, and I want to know if being in the CONTROL or EXPERIMENT groups made a difference in this relationship.

I know I can't use ANOVA, because the data results are CORRECT or INCORRECT, not normally distributed data. I thought maybe I could use Pearson's Chi-Squared test, but that only seems to deal with one factor at a time. I looked at the Cochran–Mantel–Haenszel test (the example given here: http://udel.edu/~mcdonald/statcmh.html ) but I can't seem to figure out if I can make it fit my question.

This has to be a common question. I'm trying to help someone who conducted an experiment with this, but I have very limited stats experience. Any help is appreciated.

This sounds like a good case for using Fisher's Exact Test. It sounds like you could assign your data into a 2x2 contingecy table. If that is the case, I would read up on the Tea Drinker Exp w/r Fisher's Test.

           Question A:
Question B Control Treatment Totals
Control     2      4         6
Treatment   5      6         11
Totals      7      10        17


p-value = R1!*R2!*C1!*C2! / { (grand total)! times (factorials of internals) }

p-value = (6!*11!*7!*10!) / (17!*2!*4!*5!*6!)

One more thing, MH test is a more specific example of the the Fisher's Test when used with multiple confounding levels, for example: 2x2xk matrices. Also I would not rule out the chi-square test but it assumes that you data is normal and this can be done but the warning is that you must have a large data set for that assumption of normality. The Fisher's Exact Test does not depend on the assumption of normality, ie good for small data sets.

• Thanks! I'll take a look. – mpettis Mar 26 '13 at 1:03
• As I thought about the other day yes you could chi-square if you discordant pairs are greater than 10. – oaxacamatt Mar 26 '13 at 13:29