Hi Reid,

This is a fabulous idea. In fact, it is in this line that we put out all our results so far on the WP:ACST page, though we never thought of something so ambitious. Here are my responses to your five steps:

1. Create a public Mediawiki instance: I take it that your group is handling this? I have a server I could contribute, but we probably wouldn't be able to handle the server admin. 2. Decide on a relatively standardized format of reviewing each paper (metadata formats, an infobox, how to write reviews of each, etc.): In our initial thread deciding this project, we listed a bunch of research questions; this could be used as a draft of some key aspects of papers that researchers would like to know. 3. Upload your existing Zotero database into this new wiki (I would be happy to write a script to do this): We'll be glad to share our Zotero database as you suggested. We could export it as Zotero RDF XML (which would save almost all the data, though we have a huge bank of PDFs as well), or as any format into which Zotero exports. 4. Proceed with paper readings, with the goal that every single paper is looked at by human eyes: Comments on this below. However, we hope that the results of our literature review could give a major boost to this project. 5. Use this content to produce one or more review articles: Comments on this below

The stickiest part of this project, though, has to do with authorship issues. In fact, so sticky and so important that I'll continue it on another thread. However, I am very interested in such a collaboration. And I fully agree with you that it is sufficiently important to all of us that it needs to go forward in some form, even if all authorship issues are not yet resolved.

And yes, I'm subscribed to the list, as are Arto, Mohamad and Mostafa. I've been a silent lurker for a couple years now :-). So, no need to CC me separately.

Thanks, Chitu

-------- Message original -------- Sujet: Proposal: build a wiki literature review wiki-style (was: Re: Wikipedia literature review - include or exclude conference articles) De : Reid Priedhorsky reid@reidster.net Pour : Research into Wikimedia content and communities wiki-research-l@lists.wikimedia.org Copie à : Chitu Okoli Chitu.Okoli@concordia.ca Date : March-16-11 12:20:47 PM

Chitu and others,

I too see great need for a comprehensive survey paper in this field. My own personal interest is in one that covers wiki research in general, not just research of Wikipedia; this of course makes the intractable number of papers even more intractable.

In fact, I am involved with a team of researchers with the same goal as you, though we are just getting started.

It seems to me that you are in a very difficult position. As others have noted, the scoping filter you propose is not a good one, but the number of papers is simply intractable without a very aggressive filter that excludes 2/3 or more of the known papers. (To further complicate the issue, I am skeptical of machine filtering period, fearing that any useful filter would necessarily be complex and difficult to justify in a writeup.)

However, I believe that there is a solution, and that is to dramatically increase the team size by doing the analysis wiki style. Rather than a small team creating the review, do it in public with an open set of contributors. Specifically, I propose:

1. Create a public Mediawiki instance.
2. Decide on a relatively standardized format of reviewing each paper (metadata formats, an infobox, how to write reviews of each, etc.)
3. Upload your existing Zotero database into this new wiki (I would be happy to write a script to do this).
4. Proceed with paper readings, with the goal that every single paper is looked at by human eyes.
5. Use this content to produce one or more review articles.

The goals of the effort would be threefold.

• Create an annotated bibliography of wiki research that is easy to keep up to date.
• Identify the N most important papers for more focused study and synthesis (perhaps leading towards more than one survey article).
• Provide metadata on the complete set of papers so that it can be described statistically.

Simply put, I believe that we as modern researchers need to be able to build survey articles which analyze 2,000-5,000 or more papers, and maybe this is a way to do that.

I and the other members of my team have already planned significant time towards this effort and would be very excited to join forces to lead such a mass collaboration.

Why use Mediawiki rather than Zotero or some other bibliography manager? First, it would be easy for anyone to participate because there is no software to install, no database to import, etc. Second, I personally have found Zotero, CiteULike, and every other bibliography manager I've tried to be clunky and tedious to use and not flexible enough for my needs (for example, three-state tags that let us say a paper has, does not have, or we do not know if it has, a certain property could be useful). We can always export the data into whatever bibliography software is preferred by particular authors.

Authorship is of course an issue, and one that should be worked out before people start contributing IMO, but not an intractable one, and there is precedent for scientific papers to have hundreds of authors (and it would certainly be in the wiki spirit). I myself would love to have a prominent place in the author list, but having the survey article written at all is a much higher priority.

Finally, one of my dreams has been to create a more or less complete database of *all* scientific publications, with reviews, a citation graph, private notes, and a robust data model (e.g., one that can tell two John Smiths apart and know when J. Smith is the same as John Smith). Maybe this is the first step along that path. (I did work a bit on data models for citation databases a bit about five years go and still use the software I created - Yabman, http://yabman.sf.net/.)

Thoughts?

Reid

p.s. Chitu, do you subscribe to this list? If so, we'll stop CC'ing you; if not, I encourage you to do so - it's pretty low traffic and certainly relevant to your work.