Thursday, February 12, 2015

Multiple-choice tests - all guesswork?

Multiple-choice test formats When I started in this business, MCQs were regarded as beyond the pale. Now they are increasingly de rigeur as the only way to cope. So, discuss:
MCQs: necessary evil or spawn of Satan?
No, wait, don't - we'll be here all day. Use MCQs or don't use MCQs, it's up to you. But if you use them, at least know what you're doing. Which is where this rather good short review comes in useful.

Reducing the need for guesswork in multiple-choice tests. (2014) Assessment & Evaluation in Higher Education, 40(2): 218-231, doi: 10.1080/02602938.2014.902192
The humble multiple-choice test is very widely used within education at all levels, but its susceptibility to guesswork makes it a suboptimal assessment tool. The reliability of a multiple-choice test is partly governed by the number of items it contains; however, longer tests are more time consuming to take, and for some subject areas, it can be very hard to create new test items that are sufficiently distinct from previously used items. A number of more sophisticated multiple-choice test formats have been proposed dating back at least 60?years, many of which offer significantly improved test reliability. This paper offers a new way of comparing these alternative test formats, by modelling each one in terms of the range of possible test taker responses it enables. Looking at the test formats in this way leads to the realisation that the need for guesswork is reduced when test takers are given more freedom to express their beliefs. Indeed, guesswork is eliminated entirely when test takers are able to partially order the answer options within each test item. The paper aims to strengthen the argument for using more sophisticated multiple-choice test formats, especially for high-stakes summative assessment.

No comments:

Post a Comment