[snip] The most controversial point is clearly "comprehensiveness" because you have to be an expert in your field in order to assess this, and there will be a number of opinions in most cases on what "comprehensiveness" will mean.
Hi Jürgen,
Yes, and some of the people who rated the Public Policy articles written by the students were experts. Here's how we did it:
Amy Roth, our Research Analyst, recruited a group of volunteers for the assessment. This group is made up of both policy experts and experienced Wikipedians. The policy experts all have graduate degrees in public policy or a related field from top universities, including: Berkeley, Boise, Brandeis, Georgetown, Pennsylvania, Pepperdine, Sacramento, and Syracuse. Policy experts were recruited through alumni email lists of many universities. Wikipedians were recruited through direct invitation on their user talk pages and were selected for quality assessment experience and interest in public policy topics. In the fall there were six policy expert and fifteen Wikipedian assessors (1:2.5, PE:WP). This spring there are twenty-five policy experts and eleven Wikipedians (2.3:1, PE:WP). Over the course of both terms there have been a total of twenty-eight policy experts and seventeen Wikipedians on the assessment team.
Please let me know if you have further questions,
Frank