Thursday, August 29, 2013

Condensed Milk And The Flipped Classroom

Condensed Milk In olden days a glimpse of Socky was looked on as something shocking now heaven knows - anything goes.

In a few weeks time I start teaching the first year key skills module that I have been delivering for well over a decade. This is the last time this course is scheduled to be delivered. Since the outset over 10 years ago, this course has used a flipped classroom model: students study online content including recorded mini lectures before we use all the face-to-face contact time in discussion with students and for assisting them in resolving the technical and mathematical problems they face in this course. The model has been a great success - it's one of the most efficient courses in terms of staff time that we deliver, and the students do exceedingly well in the assessments. In the past I've had complaints that the marks are "too high" and that they bias the overall marks for the year.

Yesterday I sat in yet another meeting where, after further lengthy discussion, we eventually decided that the flipped classroom could not work and that our students would not be able to cope with it. So we plan to back away from ideas of flipping our classroom in favour of giving better traditional lectures. So be it.

The Atlantic has just published the best article on the flipped classroom I have ever read: The Condensed Classroom. I would suggest that this is the only article on the flipped classroom that you need to read. We delude ourselves into thinking the flipped classroom is a new idea. As Bogosts's article points out, flipping is far from new - in fact it represents a return to ancient ideas of education which preceded the current industrialized model designed to cram ever more students into small spaces.
"As ed-tech learning practices become commonplace, we would do well to remember that technology does not improve some underlying, pure nature of their subject. Rather, it changes those things, transforming them into something new, something different. The telephone doesn't improve communication; it alters it. Facebook doesn't improve socialization; it alters it. When it comes to the process of condensation, blanket statements slip through our fingers. Condensed milk isn't necessarily worse or better than fresh milk. Winnie the Pooh likes it. It can be spread on toast or dolloped atop New Orleans snowballs. But it is not an improvement over fresh milk. It's something else entirely. Likewise, the condensed classroom ought not to be thought of as an evolution. Instead, we should see it just for what it is: one approach to learning whose merits are hardly sufficiently justified by its correspondence with current trends in Internet culture." The Atlantic

Am I sad to see my "innovative" flipped courses disappear? At first I was, but I now feel they have had their time. I don't intend to stop "innovating" or producing "novel" online learning chunklets. And I'm all in favour of giving better traditional lectures.

Lack of quantitative training among early-career ecologists

Barraquand et al. (2013) Lack of quantitative training among early-career ecologists: a survey of the problem and potential solutions. PeerJ PrePrints 1:e53v1 doi: 10.7287/peerj.preprints.53v1
Proficiency in mathematics and statistics is essential to modern ecological science, yet few studies have assessed the level of quantitative training received by ecologists. To do so, we conducted an online survey. The 937 respondents were mostly early-career scientists that studied biology as undergraduates. We found a clear self-perceived lack of quantitative training: 75% are not satisfied with their understanding of mathematical models; 75% feel the level of mathematics was “too low” in their ecology classes; 90% wanted more mathematics classes for ecologists; and 95% more statistics classes. Respondents thought that 30% of classes in ecology-related degrees should be focused on quantitative disciplines; likely more than what is taught in most universities. The main suggestion to improve quantitative training was to relate theoretical and statistical modeling to applied ecological problems. Improving quantitative training will require more mathematics classes for ecology-related degrees, and also more ecology classes containing mathematical and statistical examples.

Tuesday, August 27, 2013

Towards a student-centred definition of feedback

The more I think about feedback (which is quite a lot recently), the less interested I am in pushing mechanical feedback to students as a one way flow. Yet with increasing student numbers, we seem to be going backwards rather than forwards in having the two-way conversations that elucidate misunderstanding and can be the only really useful form of interaction between students and teaching staff. I certainly intent to target my feedback if the coming academic year using the proven checklist system I blogged about last week (Working smarter not harder with feedback).

The feedback crisis is not helped by students clearly not know what feedback is, or recognizing when they receive it (Bevan, R., Badge, J., Cann, A., Willmott, C., & Scott, J. (2008) Seeing Eye-to-Eye? Staff and Student Views on Feedback. Bioscience Education, Volume 12, doi: 10.3108/beej.12.1).

Scott, S.V. (2013) Practising what we preach: towards a student-centred definition of feedback. Teaching in Higher Education, 1-9. 19 Aug 2013 doi: 10.1080/13562517.2013.827639
Students appear to have an almost insatiable appetite for receiving feedback and the scholarly literature has acknowledged its central importance for learning. And yet there is no widely accepted definition of feedback, most definitions reflecting the perspective of the teacher rather than student. When staff at the University of New South Wales who had put a lot of time into providing feedback nevertheless failed to score highly on the course satisfaction survey question on feedback, staff conjectured that their students might not recognize what they are providing as constituting feedback. A study was undertaken to find out just how students would define feedback. This article provides the background to the study, describes its design and presents the definition of feedback as conceptualised by the students, and then considers its significance.

"Students seem to want virtually continuous feedback", not the jerky, disjointed one-way experience they receive at present.

Friday, August 23, 2013

Style in science communication

Style in science communication

"The didactic and public exposition of science is no longer, as in Kuhn’s theory, a mere static and petrified page, written by the winners in the struggle to establish a new scientific paradigm."

"in Italy there is no law, nor any other compelling moral arrangement forbidding cappuccino after lunch, but every Italian knows this should not be done."

Massimiano Bucchi. Style in science communication. Public Understanding of Science 22 August 2013 DOI: 10.1177/0963662513498202

Six Memos for the Next Millennium, Italo Calvino, 1988

Thursday, August 22, 2013

Working smarter not harder with feedback

Feeback checklist Earlier today Duncan wrote about feedback and said:

"Feedback mark one isn’t sustainable [dumb feedback as a behaviourist pedagogy which assumes that not only is the information given sufficient to actually produce a change, but that it’s unambiguous and that students will actually make use of it]. First, it absorbs a huge amount of time and resources, but its effectiveness is questionable in actually influencing student behaviour and improving student outcomes, since students may simply look at the mark and ignore the carefully considered comments. Secondly, if there is a dual emphasis on student improvement and on improving NSS scores then the problem is that students don’t recognise the feedback they receive as actually being feedback. Thirdly, mark one feedback isn’t fit for purpose since it doesn’t equip students for life post-graduation." The Culture of Student Feedback

And he's quite right. We are chasing our tails in an escalating crescendo of never enough feedback by taking the wrong turning and pushing feedback to students rather than cultivating demand for targeted feedback. Such a change is not easy to implement, but there are plenty of models which employ much smarter use of feedback than our current blanket default setting. Here's one I particularly like, partly because it has over a decade of evidence to back up its effectiveness:

Feeding forward from summative assessment: the Essay Feedback Checklist as a learning tool. (2013) Assessment & Evaluation in Higher Education 1-10, doi:10.1080/02602938.2013.822845
Owing to the increasing diversity of assessments in higher education, feedback should be provided to students in a format that can assist future and alternative work. This study aimed to assess the effectiveness of the Essay Feedback Checklist on future alternative assessments. Participants were assigned to one of two groups, one of which completed the checklist prior to assessment 1 (essay) and received feedback using this method. Attainment on assessment 1 and assessment 2 (examination) were taken as pre- and post-test scores. Results revealed increased assessment scores for the checklist group, compared to those who received conventional feedback. Focus group data indicated that students particularly liked elements of the checklist as a feedback method, but potential drawbacks were also highlighted. Implications and future use of the checklist is then discussed.

The essay feedback checklist (EFC) (Norton, L. S., & Norton, J. C. W. (2001) Essay Feedback: How Can It Help Students Improve Their Academic Writing?; Norton, L. (2011) The essay feedback checklist (EFC). A simple tool for: i) helping students write better essays, ii) targeting tutor feedback more effectively, and, iii) providing research data) works like this:

Students are asked to submit a cover sheet with their work which says:
Before you hand in your essay, please give a rating of how confident you feel about having met each of the assessment criteria:

C = Confident - I think I have met this criterion to the best of my ability
P = Partially confident – I have tried to meet this criterion but would appreciate more feedback
N = Not at all confident – I do not understand this criterion and need more guidance
  • Addresses the question
  • Clearly organized with structure appropriate to the question
  • Quality and relevance of the argument
  • Synthesis of a range of material into a coherent whole
  • Depth of understanding in relation to underlying psychological issues
  • Evaluation of theoretical concepts and research evidence
  • Appropriate academic writing style
  • Correct referencing

After marking the essay, tutors give feedback which focuses on mismatches between student self-assessment and actual performance.

One reason I particularly like this approach is because of the use of checklists. Most other professions have adopted such systems now although HE seems strangely reluctant to do so, although I have seen their effective use at other institutions. I plan to give this a try during the coming year, although I have to figure out how it fits in with the use of audio feedback, which I also plan to continue using. Any suggestions?

Wednesday, August 21, 2013

An Ecosystem of Scholarly Publishing

"Scholarly publishing in Ecology and Evolutionary Biology [EEB] is largely limited to a single species of product, the narrative paper. This monoculture of scholarly production is curious, particularly given that an entire subfield of ecology is devoted to the demonstration that diversity can lead to higher levels of function in the world’s ecosystems. The narrative paper artifact is a product of print publication, and is increasingly seen as only one vehicle for science knowledge and practice to emerge in a digital world. ... It is time for EEB to move beyond the devaluation of a alternative types of scholarly product and embrace the larger ecosystem of scholarly products."

Byrnes et al. (2013) The four pillars of scholarly publishing: The future and a foundation. PeerJ PrePrints 1:e11v1
With the rise of electronic publishing and the inherent paradigm shifts for so many other scientific endeavours, it is time to consider a change in the practices of scholarly publication in Ecology and Evolutionary Biology. To facilitate the speed and quality of science, the future of scholarly communication will rest on four pillars - an ecosystem of scholarly products, immediate and open access, open peer review, and full recognition for participating in the process. These four pillars enable us to build better tools to facilitate the discovery of new relevant work for individual scientists, one of the greatest challenges of our time as we cope with the current deluge of scientific information. By incorporating these principles into future publication platforms, we argue that science and society will be better served than by remaining locked into a publication formula that arose in the 1600s. It has served its purpose admirably and well, but it is time to move forward. With the rise of the Internet, scholarly publishing has embraced electronic distribution. But the tools afforded by the Internet and other advancing technologies have profound implications for scholarly communication beyond just distribution. We argue that, to best serve science, the process of scholarly communication must embrace these advances and evolve. Here we consider the current state of the process in ecology and evolutionary biology and propose directions for change. We identify four pillars for the future of scientific communication: (1) an ecosystem of scholarly products; (2) immediate and open access; (3) open peer review; and (4) full recognition for participating in the process. These four pillars will guide the development of better tools and practices for discovering and sharing scientific knowledge in a modern networked world. Things were far different when the existing system arose in the 1600s, and though it has served its purpose admirably and well, it is time to move forward.
This is a work in progress. Please feel free to comment at

Tuesday, August 20, 2013

Google Scholar eats away at PubMed

Google Scholar just keeps getting better, but whenever I show someone how good it now is and all the great facilities it has, they are always surprised. Most undergraduates figured this out a while back. This paper further chips away at the PubMed sacred cow.

Retrieving Clinical Evidence: A Comparison of PubMed and Google Scholar for Quick Clinical Searches. J Med Internet Res. Aug 15 2013; 15(8): e164. doi: 10.2196/jmir.2624
Physicians frequently search PubMed for information to guide patient care. More recently, Google Scholar has gained popularity as another freely accessible bibliographic database. We surveyed nephrologists (kidney specialists) and provided each with a unique clinical question derived from 100 renal therapy systematic reviews. Each physician provided the search terms they would type into a bibliographic database to locate evidence to answer the clinical question. We executed each of these searches in PubMed and Google Scholar and compared results for the first 40 records retrieved (equivalent to 2 default search pages in PubMed). We evaluated the recall (proportion of relevant articles found) and precision (ratio of relevant to nonrelevant articles) of the searches performed in PubMed and Google Scholar. Primary studies included in the systematic reviews served as the reference standard for relevant articles. We further documented whether relevant articles were available as free full-texts.
Compared with PubMed, the average search in Google Scholar retrieved twice as many relevant articles (PubMed: 11%; Google Scholar: 22%; P<.001). Precision was similar in both databases (PubMed: 6%; Google Scholar: 8%; P=.07). Google Scholar provided significantly greater access to free full-text publications (PubMed: 5%; Google Scholar: 14%; P<.001). For quick clinical searches, Google Scholar returns twice as many relevant articles as PubMed and provides greater access to free full-text articles.

Friday, August 16, 2013

An end of books?

xxx I've just finished the most difficult book I have ever read.
Every time I've been to Dublin I've wandered around thinking I really should read Ulysses one day. This time I cracked, walked into a bookshop and parted with €3 to emerge clutching a copy of Ulysses. It's taken me a few weeks but I've just finished it. Could I say that reading Ulysses was fun? Well not all the time, and I had to resort to setting myself a daily reading target in order to ensure that I kept going and didn't get distracted by, oh well, you know ... life. Do I regret reading it? Not at all, and I suspect I will become more and more glad that I did as the years go by (and certainly the next time I go to Dublin).

Last year, I read Moby Dick for the first time. That was a little different to Ulysses as I downloaded it from Project Gutenberg and read it on my iPad on long train journeys. So which of these books did I enjoy more, and did the distribution technology play any role in that? I certainly found Moby Dick easier to read than Ulysses, although of course there are frustrations when Melville goes off on one of his narrative rambles, but I couldn't say that I enjoyed it more. And I don't feel that reading a printed book versus using an iPad affected my feelings in any way, even though I find the act of reading the printed version more pleasurable than squinting at an iPad screen. I'm perfectly happy to have parted with €3 for my copy of Ulysseys, a perfectly respectable price for a book of this age by a dead author. But I am price driven, and apart from the fact that I prefer reading longform in print, the reason that I hardly ever buy ebooks is the pricing ripoff. I've got no problem shelling out €5-10 for a recent ebook by a living author, but unless I can share it with my friends and family if I enjoy reading it, then forget it. If I've bought your book, I'll damn well decide what to do with it once I've read it. My thinking now is that I just need to read War and Peace and then I'm done - I'll never need to read another book again*. And when I do read it, it will certainly be the Gutenberg version.

Seth Godin has just published an interesting post: An end of books. In some sense, every book you read is an end, as well as a beginning. I'm more interested in Godin's discussion of technology as a multitasking distraction than I am in the rest of his post. It's not an end of books we should be thinking about, but an end of longform reading. Yesterday I sat in a meeting discussing the idea of producing ebooks to support student maths teaching. I argued strongly against the original plan to produce a maths ebook-to-end-all-ebooks and in favour of shorter, topic-focused units - and algebra ebook, a calculus ebook, etc. This morning my latest purchase from Amazon arrived in the post. I will enjoy reading it. There will be no end of books. How we reading them will continue to change.

* This is a joke. I am prone to making these.

Thursday, August 15, 2013

Facebook makes you sad

facebook Or does it?

One thing that is beyond dispute is that facebook users are sad losers. But does using facebook make you sad or do sad people use facebook? The best evidence on this so far comes from a new paper in PLOS ONE:

Facebook Use Predicts Declines in Subjective Well-Being in Young Adults. (2013) PLoS ONE 8(8): e69841. doi:10.1371/journal.pone.0069841
Over 500 million people interact daily with Facebook. Yet, whether Facebook use influences subjective well-being over time is unknown. We addressed this issue using experience-sampling, the most reliable method for measuring in-vivo behavior and psychological experience. We text-messaged people five times per day for two-weeks to examine how Facebook use influences the two components of subjective well-being: how people feel moment-to-moment and how satisfied they are with their lives. Our results indicate that Facebook use predicts negative shifts on both of these variables over time. The more people used Facebook at one time point, the worse they felt the next time we text-messaged them; the more they used Facebook over two-weeks, the more their life satisfaction levels declined over time. Interacting with other people “directly” did not predict these negative outcomes. They were also not moderated by the size of people's Facebook networks, their perceived supportiveness, motivation for using Facebook, gender, loneliness, self-esteem, or depression. On the surface, Facebook provides an invaluable resource for fulfilling the basic human need for social connection. Rather than enhancing well-being, however, these findings suggest that Facebook may undermine it.

Not only did Facebook use predict a drop in happiness - people tended to be sadder by the end of every visit to the Facebook site - it also predicted a drop in people’s satisfaction with life slightly over the course of the study. That prediction held up even after controlling for differences in the frequency of real-world contact, the size of people’s Facebook networks, degree of loneliness, and self-esteem (ScienceShot).
I don't feel that the findings of this study are relevant only to facebook, I suspect they are also strongly relevant to other social networks such as twitter where users tend to share personal information and engage in online exchanges.

Of course, there are flaws in this study, with only examined 82 mid-western Americans. Most worryingly, all the participants were volunteers responding to posters asking about facebook use, so the sampling protocol is not very comprehensive.

So does facebook make you sad? We still don't know for sure.

Nice writeup (as ever) from NHS Choices here.

Tuesday, August 13, 2013

I still don't get "apps"

MyPad Maybe this is why:
most websites are fairly usable on tablets and need only limited adjustments to suit this environment. (In contrast, using websites on mobile phones requires many more design changes to accommodate the smaller screens)

when we asked people how they use their tablets, web browsing was universally mentioned as a top activity.

Should you build an app?
we advise most companies to stick to their website and invest the resources in improving web usability

You insist?
Build a tablet app only if you can offer value-added functionality over a website, such as creating an app that is focused on supporting a single main task.
don’t make your tablet app a scaled-up phone app.

Jakob Nielsen’s Alertbox: August 5, 2013: Tablet Usability

Wednesday, August 07, 2013

A glimmer of hope on the MOOC horizon

EDUC115N My pervading sense of weariness with MOOCs continues, but in spite of that, I keep signing up for new courses, and it's possible I may even complete one soon.

In some ways I feel I've had enough MOOCs to last me for a few years, but I still want to pick up ideas from other teachers to use in my own practice, so I keep checking in with new MOOCs even though I have little intention of completing most or indeed any of them. Last week I was hopeful about the Wikipedia School of Open course which has just started, but when I looked at it earlier in the week, I was immediately put off by the rigidity of the structure ("you must join a team of others straight away") and the chunkiness and lack of clarity of the Wikipedia-based environment (and the fact that the Blackboard Collaborate introductory session doesn't work on my computer didn't help either), so I binned it immediately (shame).

In contrast, the glimmer of hope is Stanford's EDUC115N How to Learn Math course, which continues to be a beacon for how MOOCs should run. I am persisting with this course where I have quickly dropped all the others I have registered for. Why is this? Well it's not the software, which although fairly straightforward to use once you learn what all the cryptic navigation symbols with no text labels mean, is still a bit clunky - particularly for submitting each assignment. And it's not the learning design - I've got short video fatigue, having watched around 80 videos so far and I'm only 70% of the way through the course (although credit for not trying to replicate a classroom lecture type experience online with a long video).

What keeps me going with EDUC115N (as I like to call it) is the approach to learning - not focused on assessment but on a clear exposition of key concepts (well supported by research evidence if you want to follow it all up). It would be easy to "cheat" your way though this course if you wanted to as the peer marking system is non-secure, but it is innovative compared to all the other MOOCs I have experienced in downplaying quantitative aspects of assessment and emphasizing engagement though activity and expression of personal opinions - and hardly an MCQ in sight.

I may have some more ideas about how we could use a "semi-peer" approach to text marking in online assessment locally tomorrow.

Tuesday, August 06, 2013

More reasons why asking students to write essays for assessment is evil

Blooms pyramid Although this paper is about essay writing by psychology students rather than science students, the conclusions remain true. Rather than using essays to construct artful and original arguments (because they'd probably be penalized by the marking scheme for doing that), students simply imitate what they've read elsewhere. Science students turn in strings of loosely connected "facts" without significant argument, viewpoint, or synthesis. It's all a long way from the top end of Bloom's pyramid. And of course, none of this treadmill activity has anything to do with what goes on in the workplace.

So let's "ban" the essay - by which I mean ask students to write sensibly structured academic reports so that they emerge from tertiary education with a transferable skill they will need.

Academic essay writing as imitative problem solving: examples from distance learning. (2013) Assessment & Evaluation in Higher Education 39(3): 1-12. doi:10.1080/02602938.2013.822846
Students in tertiary education are often faced with the prospect of writing an essay on a topic they know nothing about in advance. In distance learning institutions, essays are a common method of assessment in the UK, and specified course texts remain the main sources of information the students have. How do students use a source text to construct an essay? The present paper presents a methodology for mapping the source text on to the finished student essay. The underlying assumption is that students are using a form of imitative problem solving when faced with the complex task of writing an essay. Twenty-two essays written by Open University students in the UK, based on three different questions, were analysed on the basis of the order in which novel concepts were introduced and the extent to which this order mirrored that of the source textbook. Correlations were then carried out between the structure of the essay, the structure of the source text and the eventual grade awarded. The average correlation for all three essays and source texts was 0.8, with some individual essays having a correlation of 0.98, demonstrating that the students were closely imitating the argument structure of the source text.

Friday, August 02, 2013

Wikipedia: School of Open course

Wikipedia One of the tragedies of Wikipedia has been that academics shunned it rather than embracing it, thus acting out the tragedy of the commons. Unfortunately, contributing to Wikipedia is not as simple as it should be. If you've ever considered contributing to Wikipedia, I'd urge you to engage with an online course: Writing Wikipedia Articles: The Basics and Beyond which starts on 6th August.

Thursday, August 01, 2013

Authentic assessment

Apart from over assessment, one of the biggest problems in education is lack of authentic assessment - relevant to real world outcomes and engaging for students. But when you scratch the surface, defining what authentic assessment is - let alone designing and implementing it - is not simple. That's why I was interested in the following paper:

Establishing the critical elements that determine authentic assessment. (2013) Assessment and Evaluation in Higher Education (39(2): 1-18. doi: 10.1080/02602938.2013.819566

In this paper, the authors list eight elements they regard as critical to authentic assessment:

Authentic assessment should be challenging - tasks which establish connections between real-world experiences and school-based ideas.

The outcome of an authentic assessment should be in the form of a performance or product - does that mean that MCQs can never be authentic assessment? If so, farewell to most of e-learning.

Authentic assessment design should ensure transfer of knowledge - the extent to which the skill, knowledge and attitude being assessed may have meaning beyond the confines of a single content area.

Metacognition as a component of authentic assessment - reflection is required by the learner.

A requirement to ensure accuracy in assessment performance - authentic assessment should simulate and measure a real-world test of ability rather than just match items to curriculum content.

The role of the assessment environment and the tools used to deliver the assessment task - a ‘real world’ environment rather than a hushed exam room.

The importance of formally designing in an opportunity to discuss and provide
feedback - critical to workplace performance.

The value of collaboration - indispensable in most work environments.

So my question to you is this: how authentic an assessment of students is writing an essay?