Monday, April 04, 2011

Students participation in assessed social network activity - a statistical autopsy

Graphs Our first year Biological Sciences students have just completed their key skills course. This consists of two modules, one delivered in Term 1 (scientific literature databases, Google Reader & RSS, intellectual property, Google Docs collaborative writing, manipulating numbers, units & conversions, molarities & dilutions, areas & volumes, exponents & logs) and the other in Term 2 (Microsoft Excel, descriptive statistics, normal frequency distribution, t-test, chi-square test, correlation, regression).

Running throughout both modules is an assessed task consisting of contributing to a social network, (, designed to encourage reading and discussion of current scientific literature relevant to taught modules, and a reflective aspect ("My Diary"). I've just finished marking these contributions for Term 2 (click for larger image), which shows some interesting contrasts between the two terms.

In Term 1 (median 75%), where the friendfeed contributions contribute 30% of the module marks, there is a strong negative skew, but in Term 2 (median 40%) we see more of a bimodal pattern, where the large number of high marks is moderated by a sizable number of students who do not participate in the friendfeed exercise. This occurs for a number of reasons. In some cases, students are having academic problems across the board, whereas a proportion take the decision to incur a 20% mark penalty in order not to participate. The difference between the two modules is highly significant (Wilcoxon rank sum test, p <0.001).

What can we say about student engagement on these modules (which is the subtext of this approach)? Reading the contributions, it is clear that the students who do participate are highly engaged. The "My Diary" reflective exercise also appears to be highly engaging for those who choose to participate. But what about those who find it intrusive or too time-consuming? Apart from the fact that students are encouraged to set access to their friendfeed accounts as private (subscription only) if they wish, the My Diary exercise is only a minor component of the module mark (10%), effectively allowing students to opt out, which some do.

What happens next? Our experience from previous cohorts suggest a continued participation rate of approximately 20% - but much depends on how participation is defined. Evidence shows there are a large number of non-contributory lurkers still checking into friendfeed, occasionally popping up with a question or a remark.

Is the whole thing worth while? It's cheap (cloud-based) and incredibly efficient in terms of staff time. You betcha.


  1. So how as the marking workload been? That was one of your concerns when you re-jigged it last summer wasn't it? Glad it's going well

  2. Nice - statistically significant differences in application of the assessment lever.

  3. @Stu - Workload per head is very good.

    @Jo - Too many confounding factors to single out the assessment lever as the major cause, but certainly part of the reason for the difference.