Wednesday, August 02, 2017

Making claims for assessment [Commentary]

Usain Bolt In a recent article Dylan Wiliam does what Dylan Wiliam does best – takes a sideways but highly practical look at assessment.

The starting point for his reflection is a recent article discussing models of assessment and learning (Baird, J., Andrich, D., Hopfenbeck, T. & Stobart, G. (2017) Assessment and learning: fields apart? Assessment in Education: Principles, Policy & Practice, 24:3, 317-350, DOI: 10.1080/0969594X.2017.1319337). This argues that learning is substantially driven by assessment and that theories of assessment and theories of learning have, for the most part, proceeded entirely separately. The paper argues that that if theories of assessment take into account theories of learning, assessments will somehow be more valid. This is the point that Wiliam takes issue with:

“There is no a priori reason that processes of development would be enhanced by attention to the processes by which the results of that development are assessed, nor that the processes of assessing the results of development would be improved by considering how that development took place. For example, if we consider the case of the 100 m at the Olympic Games, a theory of learning would provide insights into how people improve their sprinting ability, and would, as a result, help us improve the quality of sprinters, so that they record lower times for the 100 m at the Olympic Games. We could also look at ways of improving the accuracy of the measurement of the time taken by sprinters to run 100 m. However, the important point here is that these two processes are entirely separate. Improvements in the measurement of time do not help us improve the performance of athletes, and improvements in the performance of athletes do not contribute to measuring sprint times more accurately. The aetiology of a performance may be entirely irrelevant to the measurement of that performance.”

Wilam goes on providing good value:

“… it may be that the introduction of more authentic assessments were driven by a desire to make the assessment more closely aligned to the learning environment. But it is at least as likely that the changes were driven by a desire to extend the kinds of inferences that test outcomes would support – to address construct underrepresentation in the assessment design. In other words, the developments of assessment theory were driven by problems of assessment, not of learning. The assessment problems may have been raised by a concern for learning, but there is little evidence that that anything in learning theory prompted the changes. Indeed, the most convincing narrative is, in my view, that developments in each of these fields have been almost completely unaffected by developments in the other. Reasons for such lack of mutual engagement and influence are easy to propose. Academics work in silos, and the pressure to publish creates incentives to make incremental improvements within a narrowly defined field, not least because criteria of quality and rigour in different fields are often very different.”

“The real problem is that measures of student progress are inherently less reliable than status measures, because the former involve subtracting one number that is measured with error from another number that is measured with error (Cronbach & Furby, 1970). The problem is not that the theory is inadequate. The problem is that the theory gives answers that people don’t like.”

“At its heart, as Lee Cronbach pointed out almost 50 years ago, an assessment is a procedure for drawing inferences: ‘One validates, not a test, but an interpretation of data arising from a specified procedure’ (Cronbach, 1971, p. 447, emphasis in original). Any educational assessment is therefore a procedure for making inferences about learning.”

Dylan Wiliam (2017) Assessment and learning: some reflections. Assessment in Education: Principles, Policy & Practice, 24:3, 394-403, DOI: 10.1080/0969594X.2017.1318108

No comments:

Post a Comment