Friday, June 08, 2007

Open Source Assessment

Stephen Downes Stephen Downes has posted a long post on Open Source Assessment at Half an Hour. Assessment is the missing link in "Education2.0". Most clued in educators are happy to embrace (or think abut the problems of embracing) Web2.0 technologies in education, but an institution which awards qualifications ultimately has responsibility for quality control, and assessment is the ultimate mechanism at which QC operates in education (plus motivating less motivated students!). So I was fascinated to read this post and want to respond to a few of the points raised:
What constitutes achievement in a field? What constitutes, for example, 'being a physicist'? ... we can't find a list of competences, for example, or course outcomes, etc., that will define a physicist.
Very true. Maybe you're a physicist if you think like a physicist ... or think you are a physicist.

Professors typically 'recognize' an A paper. They don't measure the number of salient points made nor do they count spelling errors.
Sorry Stephen, this statement is naive and outdated.

What we can expect in an open system of assessment is that achievement will be in some way 'recognized' by a community. This removes assessment from the hands of 'experts' who continue to 'measure' achievement. And it places assessment into the hands of the wider community. Individuals will be accorded credentials as they are recognized, by the community, to deserve them.
I guess that's how witch doctors are awarded their M.D.'s :-)

How does this happen? It breaks down into two parts:
- first, a mechanism whereby a person's accomplishments may be displayed and observed (e-portfolios).
- second, a mechanism which constitutes the actual recognition of those accomplishments (reputation).
In still other cases, organizations - such as universities, professional associations, governments and companies - may grant specific credentials. In such cases, the person may put forward their portfolios and profiles for consideration for the credential.
Yes, but that totally undermines the first half of your post Stephen! However, UK universities are starting to make to move towards transcripts as proposed in the Burgess Report (pdf). A transcript is not a portfolio, but it's closer to one than a traditional UK degree class.

Yes, this is a very different picture of assessment than we have today. It replaces a system in which a single set of standards was applied to the population as a whole. This was an appropriate system when it was not possible for people to view, and assess, a person's accomplishments directly. No such limitation will exist in the future, and hence, there is no need to continue to judge humans as 'grade A', 'grade B' and 'grade C'.
Unless you're trying to pick a brain surgeon for your operation. Reputation just replaces other forms of qualification. Humans always have and always will judge each other. Blair or Cameron? Kennedy or Bush? We do need to crack Assessment2.0, but this post doesn't offer the answers.


  1. A criticism would be useful, however the criticisms offered in this post - "naive and outdated", "witch doctors", "totally undermine", "trying to pick a brain surgeon for your operation", "always will judge each other," - are almost completely insubstantial. It is not possible for a reader to find the point of your objection, just that you object.

  2. Thanks for commenting Stephen, and I hope I have cause you no offence. It may be that my comments were a little harsh, and maybe I didn't emphasise enough that I think you have put your finger on a very important topic, that assessment has not moved as much as other parts of learning.
    However, I stand by my comments on your interesting post. You don't think its naive to suggest that human beings will suddenly stop judging each other? I do.
    What do others think?

  3. Where in my post did I say "human beings will suddenly stop judging each other?" I do not say this at all. Where on earth could you have gotten that idea from my post?

    As for your other comments - it wasn't that they were harsh, it's that they were empty. They suggests that you have objections, but they give the reader no information as to what those objections are.

    I think that you'll find that I have offered a fair-minded discussion, one that is well-informed and well-considered, if you'll take the time to read the post more carefully.

  4. Stephen, you said:
    "No such limitation will exist in the future, and hence, there is no need to continue to judge humans as 'grade A', 'grade B' and 'grade C'."