Piotr Konieczny wrote:
Good points. But as (few threads earlier) be lack any dedicated
publication to Wikiepdia (and as far as I know Wikimania doesn't issue
peer review compilations, and is not seen as a "real" academic
conference), works on Wikipedia are submitted to various traditional
outlets, and reviewed by traditional experts - which may be really good
in their branch of academia, but probably 1) are not Wikipedians and 2)
have not read much research on Wikipedia.
I also wonder how many research pieces on Wikipedians are written by
scholars who 1) are not Wikipedians 2) do not realize that there is an
already large body of literature on the subject...
I'm not sure to what degree this is of concern. Personally, I have on
the order of 500 Wikipedia revisions and have a vague notion of how WP
policy works - certainly not expert status, but I'd guess on the order
of 95th percentile of WP users who have edited at least once. Thinking
of my colleagues who also write and review Wikipedia work, this is not
atypical. Certainly my GroupLens colleagues understand policy far better
than I.
I regularly review for CHI, CSCW, and other high-reputation
"traditional" venues, and I also regularly reject work which is lacking
in understanding of how Wikipedia works and the existing literature. I
see other reviewers doing this as well, and the associate chairs (who
manage each paper's review) are generally well-versed in Wikipedia and
have similar high standards.
In general, understanding the user community in question is a must for
any credible research, so if you can provide evidence that the flaws you
identify are leading to systematic problems in the work that is
happening, I strongly encourage you to write it up and submit it to the
traditional venues. This kind of process critique would be very well
received.
HTH,
Reid