Friday, April 29, 2016

Who needs anonymity? The role of anonymity in peer assessment

Golden Pygmy Can students reliably and fairly assess the work of peers that are known to them? A solution to this problem (if it exists) is anonymity, but what about open learning situations where anonymity is not possible? And is anonymity desirable anyway? This well conducted study shows that anonymity improves the reliability of peer marking - but it's not essential - training of markers improves outcomes to the same extent.

The role of anonymity in peer assessment. Assessment & Evaluation in Higher Education 22 Apr 2016 doi: 10.1080/02602938.2016.1174766
This quasi-experimental study aimed to examine the impact of anonymity and training (an alternative strategy when anonymity was unattainable) on students’ performance and perceptions in formative peer assessment. The training in this study focused on educating students to understand and appreciate formative peer assessment. A sample of 77 students participated in a peer assessment activity in three conditions: a group with participants’ identities revealed (Identity Group), a group with anonymity provided (Anonymity Group) and a group with identities revealed but training provided (Training Group). Data analysis indicated that both the Anonymity Group and Training Group outperformed the Identity Group on projects. In terms of perceptions, however, the Training Group appreciated the value of peer assessment more and experienced less pressure in the process than the other two groups.

Wednesday, April 20, 2016

Student motivation in low-stakes assessment

A knotty problem Although densely written in stats-speak, this is an interesting paper for all those who, like me, have failed to get many student cohorts to engage with formative assessment. The major finding of interest here is that cohort effects trump other factors, including prior mathematical knowledge. It works in some groups, not in others. What this paper is not able to sort out is whether these differences are due to the quality of teaching groups experience, or unknown (and possibly unmeasurable) stochastic factors.

Student motivation in low-stakes assessment contexts: an exploratory analysis in engineering mechanics. Assessment & Evaluation in Higher Education 19 Apr 2016 doi: 10.1080/02602938.2016.1167164
The goal of this paper is to examine the relationship of student motivation and achievement in low-stakes assessment contexts. Using Pearson product-moment correlations and hierarchical linear regression modelling to analyse data on 794 tertiary students who undertook a low-stakes engineering mechanics assessment (along with the questionnaire of current motivation and the ‘Effort Thermometer’), we find that different measures of student motivation (effort, interest, sense of challenge, expectancy of success and anxiety to fail) showed atypical correlation patterns. The nature of the correlations further varies depending on the type of test booklet used by students. The difficulty of the early items in the assessment were positively correlated with ‘anxiety’ and ‘success’, but negatively correlated with ‘interest’. In the light of our findings, we suggest that future research should systematically explore (taking into account testing conditions like test booklet design and test-item format) the implications of student motivation for achievement in low-stakes assessment contexts. Until the consequences of these processes are better understood, the validity of assessment data generated in low-stake conditions in the higher education sector will continue to be questioned. With a greater understanding of these processes, steps could be taken to correct for student motivation in such settings, thus increasing the utility of such assessments.