0
$\begingroup$

I did a survey to measure students perceived understanding of several topic at the beginning of the semester and few weeks later. The surveys are done anonymously and I cannot match the pre and post responses. I want to measure how much their understanding has improved. Here is a sample data:

   survey                 question          response count    percent
 1 pre        I.understand.methods Strongly Disagree     5 0.05813953
 2 pre        I.understand.methods          Disagree     7 0.08139535
 3 pre        I.understand.methods           Neutral    19 0.22093023
 4 pre        I.understand.methods             Agree    32 0.37209302
 5 pre        I.understand.methods    Strongly Agree    23 0.26744186
 6 post       I.understand.methods Strongly Disagree     0 0.00000000
 7 post       I.understand.methods          Disagree     3 0.12500000
 8 post       I.understand.methods           Neutral     4 0.16666667
 9 post       I.understand.methods             Agree     8 0.33333333
10 post       I.understand.methods    Strongly Agree     9 0.37500000

The data is displayed here

It is safe to assume that the number of pre and post responses will be almost the same. In the data above it is not because I am still waiting on responses from the post survey.

What are possible metrics that I can use?

This question is related to How do I compare student pre-test scores with post-test scores to evaluate whether or not they "learned"? but different.

$\endgroup$
1
  • $\begingroup$ For future studies of this kind, you could ask each student to give info such as mother's and father's birthday and last 2 digits of driving licence number on each survey. That way you could pair (almost) all without compromising confidentiality. // For the current data, I don't immediately see a valid analysis. Indep conditions of chi-sq $2 \times 5$ contingency table (Pre/Post by Opinion) would be not be met. // If graph is for all subjects, I'd hesitate to look for clear conclusion: Disagree and Strongly Agree both increased. // Also, not measuring improvement, but perception of it. $\endgroup$
    – BruceET
    Sep 10, 2016 at 17:48

1 Answer 1

Reset to default
1
$\begingroup$

Since I haven't gotten any answers here is a solution I found. I can use the mean rank to measure how the rankings in the pre and post differ. The mean rank can be calculated from the data. For the first survey: $$ (5*1 + 7*2 + 19*3 + 32*4 + 23 * 5) / (5 + 7 + 19 + 32 + 23) = 3.71 $$

For the second survey: $$ (0*1 + 3*2 + 4*3 + 8*4 + 9*5) / (0 + 3 + 4 + 8 + 9) = 3.96 $$

So, I can conclude that the ranking has improved by $3.96 - 3.71 = 0.25$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.