Wednesday, December 10, 2014

Peer Assessment - Six is the Magic Number

Moon Peer assessment of high value written assignments? That pretty much describes this week. So this is of interest:

How to achieve accurate peer assessment for high value written assignments in a senior undergraduate course. Assessment & Evaluation in Higher Education 09 Dec 2014 doi: 10.1080/02602938.2014.987721
The psychometric measures of accuracy, reliability and validity of peer assessment are critical qualities for its use as a supplement to instructor grading. In this study, we seek to determine which factors related to peer review are the most influential on these psychometric measures, with a primary focus on the accuracy of peer assessment or how closely peer-given grades match those of an instructor. We examine and rank the correlations of accuracy, reliability and validity with 17 quantitative and qualitative variables for three senior undergraduate courses that used peer assessment on high value written assignments. Based on these analyses, we altered the single most significant variable of one of the courses. We demonstrate that the number of reviews completed per reviewer has the greatest influence on the accuracy of peer assessment out of all the factors analysed. Our calculations suggest that six reviews must be completed per reviewer to achieve peer assessment that is no different from the grading of an instructor. Effective training, previous experience and strong academic abilities in the reviewers may reduce this number.

No comments:

Post a Comment