Friday, May 03, 2013

Two-stage online testing for big classes

Online testing I had a timetable malfunction at HEA STEM13 and missed Susanne's talk which I had intended to go to. Fortunately she has now published her work so everyone can see it. It's not easy to do studies of this sort and I'm very impressed by the rigour of the statistical treatment of these findings. My frustration with this sort of work is that despite continual calls for evidence-based education, when someone puts strong research such as this into the public sphere .... we still don't use it :-(

This project aimed to improve student learning by introducing online tests, which were meant to engage students, increase the time they spend out of class on educationally meaningful activities, and to provide opportunities for self-assessment and feedback. The results suggest that increasing the time on task alone (by forcing them to spend time on online tests) did not improve student learning. Only when students were guided towards a meaningful interaction with the material, learning (as measured by exam performance) improved. The prompt, specific feedback after the formative part of the online tests enabled the students to see exactly what they needed to do in order to improve their performance. Students need to make sense of what they have learned before they are ready to move on. Giving feedback to incorrect answers and confirming correct answers contributed towards empowering students to take responsibility for their own learning.

Susanne Voelkel. Combining the formative with the summative: the development of a two-stage online test to encourage engagement and provide personal feedback in large classes. (2013) Research in Learning Technology 21: 19153
The aim of this action research project was to improve student learning by encouraging more “time on task” and to improve self-assessment and feedback through the introduction of weekly online tests in a Year 2 lecture module in biological sciences. Initially voluntary online tests were offered to students and those who participated achieved higher exam marks than those who did not, but completion rate was low. Making the tests compulsory led to high completion rates, but class performance decreased, indicating that using the same assessment for formative and for summative purposes is not always beneficial for learning. Finally, these problems were resolved by introducing a two-stage approach: the first stage of each test was formative and provided prompt feedback. However, students had to achieve 80% to progress to the second summative stage of the test. The two-stage online tests led to significantly improved class performance. This novel test design ensures that students go through at least two attempts and therefore fully benefit from the learning opportunities presented by the formative stage. Two-stage online tests present the opportunity to provide regular feedback in large classes and to improve performance not only of good but also of “weak” students.

No comments:

Post a Comment