Monday, May 23, 2011


On Friday I published a graph looking at variability of marks on out first year key skills course. This looked at the variability of marks between topics by plotting the standard deviation for each topic as an error bar. I was castigated by some people for the lack of labels on the graph ;-) so here's another one which asks the question, is the difference in marks between topics statistically significant?


By plotting the standard error of the mean as an error bar, we can determine whether differences between topics are statistically significant. When SEM error bars do not overlap, we cannot be sure that the difference between two means is statistically significant, but when they do, you can be sure the difference between the two means is not statistically significant (P>0.05). In this graph, the marks for the Units & Conversions and Molarities and Dilutions topics are not just lower, but are significantly lower than the other topics, suggesting this is where more effort should be concentrated.



  1. I used to point out that the likelyhood of a significant difference for pairwise comparisons was indicated by significance space between error bar overlaps, using 2*sem as a visual estimate as, empirically, it was usually robust with student provided data (i.e. occasionally 'very dodgy'). Btw, Minitab provided nice output options to illustrate individual pairwise differences when the visual SEM gap was difficult to rely on (but I was advised to try to do everything in Excel "as that is what they will probably use after they leave". #dummingdownnotusmategiveusyour9kand...)

  2. Hmm - it's odd that the logs marks came out so well, that's usually an area where they usually struggle. In fact I had to re-read your axes labels to be sure I had read it correctly.