See which questions work, which distractors are effective, and how learners perform at a glance. These graphs are designed to show how the probability of a student choosing an answer option is related to the ability of the student.
Student ability is shown on the x-axis. The probability of them choosing an option is shown on the y-axis.
Higher-ability learners would be expected to choose the correct answer more – you can see this the case in the yellow trace.
An effective distractor should attract more lower-ability learners. There is evidence of this in the green trace. A higher proportion of low ability learners are choosing this option. Effective distractors like this build increased discrimination between higher and lower ability students. This is the main aim of assessments designed to grade students.
There is an argument to say that this question was actually slightly easy as the correct answer was often chosen by students below the average ability…notice how the gradient of the yellow line starts to increase between -0.9 and -0.4. Put another way…students with a lower than average ability are all getting this question right. This is fine for building confidence or recall practice but this is not great for discriminating between high ability and low ability students in a grading assessment.
Poor question
Q25 shows an example of a poor question or poor distractors. Notice how the traces are all quite flat. None of the options are clearly favoured by either high or low performing students. Knowing a little more about this subject area, this is likely to be a problem with the question. Magnetic tape is the conventional answer to this type of exam question but it’s a good example of exam boards not keeping pace with the real world. Most modern data centers would back up to another device containing an SSD – magnetic tape drives are museum pieces now.
This is a great example of how the graph has identified a problem with the question. As a teacher I would look to incorporate a careful discussion of this sort of this sort of exam question vs real life conflict. There is obviously learning to do here.
So on one hand, having this question in the assessment has played a valuable role, but from a discrimination and grading perspective this question should be considered quite unhelpful as it blurs the difference between low ability and high ability students.