Hi Reid,
This is a fabulous idea. In fact, it is in this line that we put
out all our results so far on the WP:ACST page, though we never
thought of something so ambitious. Here are my responses to your
five steps:
1. Create a public Mediawiki instance: I take it
that your group is handling this? I have a server I could
contribute, but we probably wouldn't be able to handle the server
admin.
2. Decide on a relatively standardized format of reviewing each
paper (metadata formats, an infobox, how to write reviews of each,
etc.): In our initial thread deciding this project, we listed a
bunch of research questions; this could be used as a draft of some
key aspects of papers that researchers would like to know.
3. Upload your existing Zotero database into this new wiki (I would
be happy to write a script to do this):
We'll be
glad to share our Zotero database as you suggested. We could export
it as Zotero RDF XML (which would save almost all the data, though
we have a huge bank of PDFs as well), or as any format into which
Zotero exports.
4. Proceed with paper readings, with the goal that every single
paper is looked at by human eyes: Comments on this below. However,
we hope that the results of our literature review could give a major
boost to this project.
5. Use this content to produce one or more review articles: Comments
on this below
The stickiest part of this project, though, has to do with
authorship issues. In fact, so sticky and so important that I'll
continue it on another thread. However, I am very interested in
such a collaboration. And I fully agree with you that it is
sufficiently important to all of us that it needs to go forward
in some form, even if all authorship issues are not yet
resolved.
And yes, I'm subscribed to the list, as are Arto, Mohamad and
Mostafa. I've been a silent lurker for a couple years now :-). So,
no need to CC me separately.
Thanks,
Chitu
Chitu and
others,
I too see great need for a comprehensive survey paper in this
field. My own personal interest is in one that covers wiki
research in general, not just research of Wikipedia; this of
course makes the intractable number of papers even more
intractable.
In fact, I am involved with a team of researchers with the same
goal as you, though we are just getting started.
It seems to me that you are in a very difficult position. As
others have noted, the scoping filter you propose is not a good
one, but the number of papers is simply intractable without a very
aggressive filter that excludes 2/3 or more of the known papers.
(To further complicate the issue, I am skeptical of machine
filtering period, fearing that any useful filter would necessarily
be complex and difficult to justify in a writeup.)
However, I believe that there is a solution, and that is to
dramatically increase the team size by doing the analysis wiki
style. Rather than a small team creating the review, do it in
public with an open set of contributors. Specifically, I propose:
1. Create a public Mediawiki instance.
2. Decide on a relatively standardized format of reviewing each
paper (metadata formats, an infobox, how to write reviews of each,
etc.)
3. Upload your existing Zotero database into this new wiki (I
would be happy to write a script to do this).
4. Proceed with paper readings, with the goal that every single
paper is looked at by human eyes.
5. Use this content to produce one or more review articles.
The goals of the effort would be threefold.
* Create an annotated bibliography of wiki research that is easy
to keep up to date.
* Identify the N most important papers for more focused study and
synthesis (perhaps leading towards more than one survey article).
* Provide metadata on the complete set of papers so that it can be
described statistically.
Simply put, I believe that we as modern researchers need to be
able to build survey articles which analyze 2,000-5,000 or more
papers, and maybe this is a way to do that.
I and the other members of my team have already planned
significant time towards this effort and would be very excited to
join forces to lead such a mass collaboration.
Why use Mediawiki rather than Zotero or some other bibliography
manager? First, it would be easy for anyone to participate because
there is no software to install, no database to import, etc.
Second, I personally have found Zotero, CiteULike, and every other
bibliography manager I've tried to be clunky and tedious to use
and not flexible enough for my needs (for example, three-state
tags that let us say a paper has, does not have, or we do not know
if it has, a certain property could be useful). We can always
export the data into whatever bibliography software is preferred
by particular authors.
Authorship is of course an issue, and one that should be worked
out before people start contributing IMO, but not an intractable
one, and there is precedent for scientific papers to have hundreds
of authors (and it would certainly be in the wiki spirit). I
myself would love to have a prominent place in the author list,
but having the survey article written at all is a much higher
priority.
Finally, one of my dreams has been to create a more or less
complete database of *all* scientific publications, with reviews,
a citation graph, private notes, and a robust data model (e.g.,
one that can tell two John Smiths apart and know when J. Smith is
the same as John Smith). Maybe this is the first step along that
path. (I did work a bit on data models for citation databases a
bit about five years go and still use the software I created -
Yabman, http://yabman.sf.net/.)
Thoughts?
Reid
p.s. Chitu, do you subscribe to this list? If so, we'll stop
CC'ing you; if not, I encourage you to do so - it's pretty low
traffic and certainly relevant to your work.