James and
Travis, you bring up a point that we have struggled back and
forth with for several months. We really, really would like to
include conference articles, but we just can't see how we could
handle many more articles than what we've got now. We've been
working on and off on this project for over two years now. (You
can find works in progress at the link at the bottom to my
website.) We'd like to get it done eventually,
and we can only handle so many articles.
We considered including top-tier conferences, but the question is,
what is a "top conference"? In trying to answer this, we looked at a
couple of sources:
* Top Tier and 2nd tier conferences from
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?page=cforsel10
* We also considered including all WikiSym articles on Wikipedia
We identified which of the 1,500 conference papers from
http://en.wikipedia.org/wiki/User:Moudy83/conference_papers were
"top conferences" by those definitions, and we found over 400. On
top of our 600 journal articles and doctoral theses, we think 1,000
papers is just too much for us to handle.
If we could somehow narrow it down to 100 relevant conference
papers, we could add that in, but no more. However, how do we select
which conferences are "must includes" while unfortunately leaving
out the rest? We just don't know how to do this in a non-arbitrary,
objective manner that would truly identify the top 100 conference
papers on Wikipedia that contribute to scholarly knowledge.
Any ideas on how to do this would be very much appreciated.
Regards,
Chitu
As
an HCI/CS researcher who has published at top peer-reviewed
conferences about Wikipedia, but not journals, I'd like to echo
James' statements. Journals are not the norm in CS/HCI research.
Knowledge is shared through conferences, not journals.
On 3/14/11 11:32 AM, James Howison wrote:
Hi
there,
Great project; massive but will be much appreciated. We did
something similar for empirical studies of Open Source, recently
accepted at ACM Computing Surveys (PDF pre-print available here
[1], article not in print until 2012 (!! that's another email
entirely, bah))
I recognize the need to cut down the number of articles for
review, we reviewed around 600 and that was a multi-year effort.
We did that mainly by excluding conceptual (hence empirical) or
passing reference articles (ie we did a two-step filter on many
more articles), but were forced to only do journal articles for
updates during the (long) revision process. I regret that
necessity, it decreases the utility of the work.
Given the publication venues of choice for many academics in
this community I do wonder if you aren't shooting yourself in
the foot by excluding peer-reviewed conferences and restricting
to journals. Personally I'd rather read a review that included
the top journals and top conferences than one that included all
journals. Or even rather read a review over a shorter time
period that included publications over journals and conferences,
or on more specified topics. The interesting question is, "what
do we know about wikipedia" not "what did we publish in journals
about wikipedia". In particular you will find you have
systematically excluded the contribution of HCI authors.
Given the commendable and massive effort you are providing (and
your approach to coverage below is really interesting), getting
that wrong at the outset seems a shame.
Best regards,
James Howison
[1] Crowston, K., Wei, K., Howison, J., and Wiggins, A. (2012).
Free (libre) open source software development: What we know and
what we do not know. ACM Computing Surveys, 44(2):
http://floss.syr.edu/content/freelibre-open-source-software-development-what-we-know-and-what-we-do-not-know
On Mar 14, 2011, at 13:58, Chitu Okoli wrote:
Hi everyone,
We are a research group conducting a systematic literature
review on Wikipedia-related peer-reviewed academic studies
published in the English language. (Although there are many
excellent studies in other languages, we unfortunately do not
have the resources to systematically review these at any kind
of acceptable scholarly level. Also, our study is about
Wikipedia only, not about other Wikimedia Foundation projects.
However, we do include studies about other language
Wikipedias, as long as the studies are published in English.)
We have completed a search using many major databases of
scholarly research. In a separate thread, we will also talk
about research questions related to our review.
Thanks for your help.
Chitu Okoli, Concordia University, Montreal, Canada
(http://chitu.okoli.org/professional/open-content/wikipedia-and-open-content.html)
Arto Lanamäki, University of Agder, Kristiansand, Norway
Mohamad Mehdi, Concordia University, Montreal, Canada
Mostafa Mesgari, Concordia University, Montreal, Canada