Tuesday, September 29, 2015

How to write an essay

Problem 1: Students write descriptive essays which do not demonstrate critical thinking.
Solution: Title as question.

Problem 2: Massive over reliance on essays as an assessment format in higher education.
Solution: ???

Henri D, Morrell L. and Scott G. Ask a clearer question, get a better answer. F1000Research 2015, 4: 901 doi: 10.12688/f1000research.7066.1
Many undergraduate students struggle to engage with higher order skills such as evaluation and synthesis in written assignments, either because they do not understand that these are the aim of written assessment or because these critical thinking skills require more effort than writing a descriptive essay. Here, we report that students who attended a freely available workshop, in which they were coached to pose a question in the title of their assignment and then use their essay to answer that question, obtained higher marks for their essay than those who did not attend. We demonstrate that this is not a result of latent academic ability amongst students who chose to attend our workshops and suggest this increase in marks was a result of greater engagement with ‘critical thinking’ skills, which are essential for upper 2:1 and 1st class grades. The tutoring method we used holds two particular advantages: First, we allow students to pick their own topics of interest, which increases ownership of learning, which is associated with motivation and engagement in ‘difficult’ tasks. Second, this method integrates the development of ‘inquisitiveness’ and critical thinking into subject specific learning, which is thought to be more productive than trying to develop these skills in isolation.

I'm not quite sure how peer review works on the F1000 education channel. I have been asked to peer review articles in the past and have done so, but I'm not sure how open the process is. So here's my open peer review of this paper.

This is an interesting and potentially valuable study of a method to improve the quality of student writing. The sample size is relatively small and the major weaknesses are pointed out by the authors in the Discussion:
"For the purpose of this study we assumed that students who posed a question in the title of their essay had attended the workshop and understood the underlying concepts of the workshop, and this has been used as the independent factor in our analysis. We acknowledge that this lack of certainty in the allocation of students to the did/did not attend category does need to be borne in mind when interpreting our results. Another possible confounding factor is that voluntary workshop attendance may be skewed towards individuals who are more engaged or motivated with the module; and these individuals are more likely to obtain higher grades because of this higher engagement with the module content"
To counteract these factors, the authors should cite an effect size to validate the p-values quoted in the results.

Thursday, September 17, 2015

The 3 P's of Feedback - again

Since I wrote about the 3 P's of good feedback a couple of weeks ago I've had several conversations with colleagues which seem to need some amplification.

Not much to add to what I said previously - I suspect that for students prompt means 21 seconds or 21 minutes after submission, not 21 days - except that electronic submission of work has changed the game and is probably responsible for some of the dissatisfaction we face. When students had to trek across campus to a printer, print their work, then track back to a departmental office to stand in a queue to hand it in, 21 days seemed like a fair interval to wait for a response. But that's not how it is any more. Clicking a Submit button online generates an expectation of a much quicker interaction. For anything other than simple automatically marked MCQs, this just isn't going to happen in the interval (seconds or minutes) that students want. I don't know what we do about this, other than to get better at explicit telling students what to expect "After submission you will receive your marks and feedback within 24 hours / one week / whenever."

Group feedback does not satisfy the feedback beast. However, it could buy us some time, if it's sufficiently prompt. Group feedback on essays (good points and common errors) within 24 hours with detailed personal feedback to follow in 7 days? Feedback on exams is another huge problem that group feedback won't solve. The examination process used to take a few days, or maybe even hours for a small class. Now it takes weeks. This is partly because of student numbers, but mostly because of the huge backend bureaucracy we have built. Students don't understand this, it further distances us from them. And yet I can't see the bureaucracy going away, so the only inadequate solution I have is to better communicate with students why it takes us so long.

This is the idea that has generated most heat in discussions. "If we don't tell students what's wrong with their work how can they improve?". I was raised in an academic era when my work was bluntly, sometime brutally, criticized (when needed). Crap was frequently scrawled in the margins, and it's hard for me to break out of that mould. It was tough at the time, but you could argue that it made me resilient (or maybe I was already resilient, which enabled me to survive). Either way, it doesn't work any more. If feedback is not positive, non-resilient students switch off and disengage. But that doesn't mean you can't tell students what's wrong with their work. Rather than writing "You didn't include any diagrams", say "If you include some diagrams in your next essay you will get better marks". One consequence of this is that the classic sh*t sandwich feedback formula is now beyond the pale. We're in the unbearably upbeat Have A Nice Day era. We need to adapt. Feedback is not peer review. Feedback is not performance monitoring, it is mentoring. Once again we have fallen down the crack between feedback and assessment.

Friday, September 11, 2015

Student use of Wikipedia as an academic resource

Wikipedia I am encouraged rather than discouraged by:

Selwyn, N., and Gorard, S. (2015) Students' use of Wikipedia as an academic resource - patterns of use and perceptions of usefulness. The Internet and Higher Education. 5 September 2015 doi: 10.1016/j.iheduc.2015.08.004
Wikipedia is now an established information source in contemporary society. With initial fears over its detrimental influence on scholarship and study habits now subsiding, this paper investigates what part Wikipedia plays in the academic lives of undergraduate students. The paper draws upon survey data gathered from students across two universities in Australia (n=1658), alongside follow-up group interview data from a subsample of 35 students. Analysis of this data suggests that Wikipedia is now an embedded feature of most students’ study, although to a lesser extent than other online information sources such as YouTube and Facebook. For the most part, Wikipedia was described as an introductory and/or supplementary source of information – providing initial orientation and occasional clarification on study topics. While 87.5 per cent of students reported using Wikipedia, it was seen to be of limited usefulness when compared with university-provided library resources, e-books, learning management systems, lecture recordings and academic literature databases. These findings were notably patterned in terms of students’ gender, year of study, first language spoken and subject of study.
  • Draws on survey data examining 1658 undergraduate students’ uses of digital technologies for academic purposes.
  • 87.5 per cent of students report having used Wikipedia to find information for their academic work, with 24.0 per cent of these considering Wikipedia to have been ‘very useful’.
  • Use and perceived usefulness of Wikipedia is most prevalent amongst students who are male, in advanced years of study, from non-English speaking households, and those studying engineering, science and medicine subjects.
  • Rather than constituting a primary source of information, students report Wikipedia mainly playing introductory or clarificatory roles in their information gathering and research.

See: Citing Wikipedia in Academic Work

Friday, September 04, 2015

Formative and shared assessment in higher education

Back to school "The aim of this article is to review the use of formative and shared assessment within higher education."

  • Setting clear learning goals that students can achieve
  • Providing students with feedback to guide them in their learning
  • Involving students in the learning process, self-evaluation and assessment
  • Promoting feedback as a process of dialogue
  • Finding a balance between the ideal time spent in formative assessment processes and conditions in which the course is developed

Formative and shared assessment in higher education. Lessons learned and challenges for the future. Assessment & Evaluation in Higher Education 03 Sep 2015 doi: 10.1080/02602938.2015.1083535
Formative assessment is the process by which teachers provide information to students during the learning process to modify their understanding and self-regulation. An important process within this is shared assessment, which refers to student involvement in the assessment and learning practice, a process of dialogue and collaboration between teacher and students aimed at improving the learning process, both individually and collectively. The purpose of this paper is to review the current state of affairs in depth. This paper therefore highlights, on the one hand, the lessons learned through research and development in higher education (i.e. providing clear learning goals and feedback, guiding learning, involving students in learning and assessment, promoting feedback as a process of dialogue, and making processes viable). On the other hand, these lessons also suggest some challenges and difficulties that must be addressed in the future in order to further improve formative and shared assessment in higher education. These include the need for more research on its effects, further conceptual clarification, the intersubjectivity of the process, recognition of the divergent processes and ethical principles, students’ involvement not only in assessment but also in determining academic grades, and broadening learning goals and objectives in FA & SA.