Tuesday, August 08, 2017

Collaborative testing promotes higher order thinking

Graduates I'm not sure I completely understand why, but the English education system installs a fierce sense of competition in students. Although we attempt to make them aware that we use criterion- rather than norm-referenced assessment, we just can't seem to convince them that assessment is not a zero sum game. Consequently, group assessment, especially summative group assessment, generates fireworks. But it turns out that collaborative testing is a valuable exercise, assuming we're interested in higher order skills rather than just cramming their heads with facts?

The effects of collaborative testing on higher order thinking: Do the bright get brighter? Active Learning in Higher Education 04.08.2017 doi: 10.1177/1469787417723243
Collaborative testing has been shown to enhance student performance compared to individual testing. It is suggested that collaborative testing promotes higher order thinking, but research has yet to explore this assumption directly. The aim of this study was to explore the benefits of collaborative testing on overall performance, as well as performance on higher order thinking questions. It was hypothesised that, compared to individual test results, students would perform better overall and on higher order thinking questions under collaborative testing conditions. It was expected that these differences would be equal when comparing students of different academic abilities (i.e. ‘upper’, ‘middle’ and ‘lower’ performers). Undergraduate students completed an individual followed by a collaborative test as part of summative assessment. Analyses revealed that with the exception of upper performers, students performed better overall on the collaborative test. Additionally, regardless of their academic abilities, students performed better on the higher order thinking questions under collaborative conditions. This improvement was equal across different academic abilities, suggesting that collaborative testing promotes higher order thinking even when taking into account previous academic achievement. The acceptability and application of collaborative testing is discussed.

Wednesday, August 02, 2017

Making claims for assessment [Commentary]

Usain Bolt In a recent article Dylan Wiliam does what Dylan Wiliam does best – takes a sideways but highly practical look at assessment.

The starting point for his reflection is a recent article discussing models of assessment and learning (Baird, J., Andrich, D., Hopfenbeck, T. & Stobart, G. (2017) Assessment and learning: fields apart? Assessment in Education: Principles, Policy & Practice, 24:3, 317-350, DOI: 10.1080/0969594X.2017.1319337). This argues that learning is substantially driven by assessment and that theories of assessment and theories of learning have, for the most part, proceeded entirely separately. The paper argues that that if theories of assessment take into account theories of learning, assessments will somehow be more valid. This is the point that Wiliam takes issue with:

“There is no a priori reason that processes of development would be enhanced by attention to the processes by which the results of that development are assessed, nor that the processes of assessing the results of development would be improved by considering how that development took place. For example, if we consider the case of the 100 m at the Olympic Games, a theory of learning would provide insights into how people improve their sprinting ability, and would, as a result, help us improve the quality of sprinters, so that they record lower times for the 100 m at the Olympic Games. We could also look at ways of improving the accuracy of the measurement of the time taken by sprinters to run 100 m. However, the important point here is that these two processes are entirely separate. Improvements in the measurement of time do not help us improve the performance of athletes, and improvements in the performance of athletes do not contribute to measuring sprint times more accurately. The aetiology of a performance may be entirely irrelevant to the measurement of that performance.”

Wilam goes on providing good value:

“… it may be that the introduction of more authentic assessments were driven by a desire to make the assessment more closely aligned to the learning environment. But it is at least as likely that the changes were driven by a desire to extend the kinds of inferences that test outcomes would support – to address construct underrepresentation in the assessment design. In other words, the developments of assessment theory were driven by problems of assessment, not of learning. The assessment problems may have been raised by a concern for learning, but there is little evidence that that anything in learning theory prompted the changes. Indeed, the most convincing narrative is, in my view, that developments in each of these fields have been almost completely unaffected by developments in the other. Reasons for such lack of mutual engagement and influence are easy to propose. Academics work in silos, and the pressure to publish creates incentives to make incremental improvements within a narrowly defined field, not least because criteria of quality and rigour in different fields are often very different.”

“The real problem is that measures of student progress are inherently less reliable than status measures, because the former involve subtracting one number that is measured with error from another number that is measured with error (Cronbach & Furby, 1970). The problem is not that the theory is inadequate. The problem is that the theory gives answers that people don’t like.”

“At its heart, as Lee Cronbach pointed out almost 50 years ago, an assessment is a procedure for drawing inferences: ‘One validates, not a test, but an interpretation of data arising from a specified procedure’ (Cronbach, 1971, p. 447, emphasis in original). Any educational assessment is therefore a procedure for making inferences about learning.”

Dylan Wiliam (2017) Assessment and learning: some reflections. Assessment in Education: Principles, Policy & Practice, 24:3, 394-403, DOI: 10.1080/0969594X.2017.1318108