In short, PeerWise is an automated self-study, low-stakes assessment
system where both questions and answers are edited and reviewed by
anyone (with access; in practice this usually means anyone enrolled in
a course or major at an institution) very similarly to textual content
in a wiki. It is already being used successfully at hundreds of higher
education and other institutions. But sadly it's closed source. I
have since 2009 been trying to encourage the Foundation to build an
open source version of such a system.
Is there anyone else interested in this?
@James: I'm intrigued by this system, and I've talked to a Wikipedian in Auckland who used it and liked it. However, I think blanking the database at the beginning of each course is a big mistake, as is limiting it to a small class audience. I imagine building a similar system that is monolithic (a single database for all topics), accepts contributions from the general public, accumulates over time like Wikipedia, and is moderated by experienced users using tags and/or a hierarchy. A sort of "Wikipedia of assessment" if you will. In principle it's even possible to incorporate short answer and essay questions by leveraging some mixture of machine learning and peer review - positive and negative examples could then be highlighted with comments to help provide feedback to others.
I'm equipped to prototype a system like this and it would mesh well with my research, but I'd like to know your thoughts, as well as if there are other interested parties you might recruit. Let me know. :-)