So I have a study that I have carried out with 40 test subjects. The study consisted of reading two different texts using different methods and answering a number of questions corresponding to each text. Half the population read text $t_1$ using method $m_1$ and $t_2$ using method $m_2$. The other half read text $t_1$ using method $m_2$ and text $t_2$ using method $m_1$.

I'm trying to figure out if there's any inherent difference in difficulty between the two texts, but I can't figure out which statistical test to use. It feels as if a paired t-test is wrong since the same person read the two texts using different methods, but I'm not really sure what else to use.

  • 1
    $\begingroup$ You have an answer here, but perhaps the question really belongs on cross validated (stats.stackexchange.com) where more people hang out who know about these kinds of thngs. $\endgroup$ – Ethan Bolker Apr 16 '17 at 18:01
  • $\begingroup$ Thank you Ethan, crossposted there aswell! $\endgroup$ – Nyfiken Gul Apr 19 '17 at 12:19
  • $\begingroup$ @nyfikengul where is your question on stats.SE? Please add a link in a comment or update this question on Math.SE with it. $\endgroup$ – unseen_rider Apr 19 '17 at 16:44

IMO you since are doing comparisons between four different groups (results sets), an ANOVA (Analysis of Variance) approach would be the best approach.

  1. Analyse and look at your data, and remove any outliers first.

  2. Check if the all the assumptions for ANOVA have been satisfied.

I will presume that the assumption of the dependent variable being normally distributed for each group is not satisfied (can be checked via anderson-darling test, and graphs of the data).

  1. Now you need to check for homogeneity of variances. One can do this via Levene's test.

$H_0: \sigma_1 = \sigma_2 = \sigma_3 = \sigma_4$ $H_1: \sigma_i \ne \sigma_j , i\ne j, i,j=(1,2,3,4)$


After obtaining the results of Levene's test:

i. If any of the p-values are greater than or equal to $0.05$ then:

  1. $H_0$ is rejected and homogeneity of variances assumption is violated, as well as normality.
  2. This would imply a nonparametric ANOVA is appropriate - such as Kruskal-Wallace. Software such as SAS, SPSS have this. MS Excel may have also.


    Read section "Reasons to Use Parametric Tests" on http://blog.minitab.com/blog/adventures-in-statistics-2/choosing-between-a-nonparametric-test-and-a-parametric-test

    If there are significant differences found between the 4 groups, you will need to do post adhoc comparisons for determining which of the four groups are significantly different from each other. Holm's t-test could be used for these.

ii. Otherwise if all p-values are less than $0.05$ then one can assume that $H_0$ is accepted - all variances of groups are the same. Then you would need to research which test to do next (with non-normal data, and equal variances).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.