New subject: Wikipedia literature review

18 Mar 2011

Hi all, after following these links:

...
  * Top Tier and 2nd tier conferences from
 http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
 * A-ranked conferences in Information and Computing Sciences from
 http://lamp.infosys.deakin.edu.au/era/?pageÏorsel10 
I discovered that CHI is not a top-tier HCI conference. Or even 2nd or
3rd. :) Neither is CSCW, Group, etc.

(For those not in CS/HCI, CHI is *the* top tier conference with
acceptance rates lower than most journals in the area and the others I
listed are nearly as competitive. And venues like WikiSym that have
higher acceptance rates attract top-tier work as well - many papers of
equivalent quality have been published there.)

So... I would suggest that, to review the Wikipedia literature, you
need to choose a field (or set of fields) you know and become deeply
knowledgeable about the literature in that field by reading it and
following citations, etc. Searching for every paper ever written that
mentions "Wikipedia" in the title/abstract will catch you lots of
peripheral work that is not *about* Wikipedia, but that uses Wikipedia
as a context for studying something else. Machines aren't good at
literature reviews. :)

That said, it would be incredibly useful to have a common repository
of citations that can be annotated, discussed etc. Reid Priedhorsky,
Phoebe Ayers, Brent Hecht, Darren Gergle, and Mako Hill and I have
been talking about doing a literature review as well and have come to
the conclusions that A) there's too much to cover in just one paper
and B) a place to collaboratively assemble knowledge about the
literature is a prerequisite for such an endeavor. We were thinking a
MediaWiki with templates to structure citation data for export would
be better than any of the bib software out there. But actually... I
remember reading that Tiki Wiki now explicitly supports citation, I
think?

The bottom line is, B) Is something this community could really do a
great job of developing and it would be mutually beneficial. No one of
us is going to cover all the Wikipedia literature alone, unless we
make it a full time job. :)

FYI, we chose Travis's citation filtering method as a starting point
and came up with the list below as our common starter reading list and
planned to divy things up from there.

Andrea

== Top Citations in Goog Scholar ==

* '''449 cites''' Fernanda B. Viegas, Martin Wattenberg, and
Kushal
Dave. 2004. [http://alumni.media.mit.edu/~fviegas/papers/history_flow.pdf
Studying cooperation and conflict between authors with history flow
visualizations]. In Proceedings of the SIGCHI conference on Human
factors in computing systems (CHI '04). ACM, New York, NY, USA,
575-582.

* '''288 cites''' Bryant, Susan, Andrea Forte and Amy Bruckman.
(2005). [http://www.andreaforte.net/BryantForteBruckBecomingWikipedian.pdf
Becoming Wikipedian: transformation of participation in a
collaborative online encyclopedia]. Proceedings of GROUP International
Conference on Supporting Group Work, Sanibel Island, FL, pp. 1-10.
** recent counterpoint: Katherine Panciera, Aaron Halfaker, and Loren
Terveen. 2009. [http://www.grouplens.org/system/files/Group09WikipediansPanciera.pdf
Wikipedians are born, not made: a study of power editors on
Wikipedia]. In Proceedings of the ACM 2009 international conference on
Supporting group work (GROUP '09). ACM, New York, NY, USA, 51-60.

* '''193 cites''' Voß, J.
[http://hapticity.net/pdf/nime2006_180-works_cited/MeasuringWikipedia2005.pdf
Measuring Wikipedia]. Proceedings of 10th International Conference of
the International Society for Scientometrics and Informetrics,
(Stockholm, Sweden), 2005.

* '''187 cites''' Lih, A.
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.9104&rep=r…
Wikipedia as Participatory journalism: reliable sources? metrics for
evaluating collaborative media as a news resource]. Proceedings of
Fifth International Symposium on Online Journalism, April 16-17, 2004,
(Austin, TX), 2004.

* '''145 cites''' Fernanda B. Viegas, Martin Wattenberg, Jesse
Kriss,
Frank van Ham,
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.6907&rep=re…
Talk Before You Type: Coordination in Wikipedia] hicss, pp.78a, 40th
Annual Hawaii International Conference on System Sciences (HICSS'07),
2007

* '''140 cites''' Kittur, A.; Chi, E. H. ; Pendleton, B. A. ; Suh,
B.
; Mytkowicz, T. Power of the few vs. wisdom of the crowd: Wikipedia
and the rise of the bourgeoisie. Alt.CHI at CHI 2007; 2007 April 28 -
May 3; San Jose, CA.

* '''138 cites''' Aniket Kittur, Bongwon Suh, Bryan A. Pendleton,
and
Ed H. Chi. 2007. He says, she says: conflict and coordination in
Wikipedia. In Proceedings of the SIGCHI conference on Human factors in
computing systems (CHI '07). ACM, New York, NY, USA, 453-462. '

* '''101 cites''' Reid Priedhorsky, Jilin Chen, Shyong (Tony) K.
Lam,
Katherine Panciera, Loren Terveen, and John Riedl. 2007.
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.123.7456&rep=r…
Creating, destroying, and restoring value in wikipedia]. In
Proceedings of the 2007 international ACM conference on Supporting
group work (GROUP '07). ACM, New York, NY, USA, 259-268.

== Other Significant Papers ==

* '''Best Paper Award''' Ivan Beschastnikh, Travis Kriplean and
David
W. McDonald [http://www.aaai.org/Papers/ICWSM/2008/ICWSM08-011.pdf
Wikipedian Self-Governance in Action: Motivating the Policy
Lens]International Conference on Weblogs and Social Media, 2008.

* '''Honorable Mention''' Travis Kriplean, Ivan Beschastnikh and
David
W. McDonald [http://dub.washington.edu/djangosite/media/papers/tmpZ77p1r.pdf
Articulations of WikiWork: Uncovering Valued Work in Wikipedia through
Barnstars] Computer Supported Cooperative Work, 2008.

== Relevant Scholarly Books ==

Reagle's book, Lih's book, Sunstein, Benkler

On Wed, Mar 16, 2011 at 7:07 PM, Chitu Okoli &lt;Chitu.Okoli(a)concordia.ca&gt; wrote:
> Hi Jack,
>
> Actually, the reason we're talking about top-tier is based on the same
> reason we talk about peer-reviewed versus non-peer-reviewed. No one can
> argue that non-peer-reviewed work (such as working papers) often have
> completely novel ideas. The problem is that someone has to wade through tens
> of thousands of works of hugely varying quality to find a few pearls. The
> peer-review process does this wading; while it might miss a few novel items,
> it would probably get most of the high-quality ones. Similarly, there are at
> least 2,000 Wikipedia studies. Since we can't go through all of them, we
> hope that most of the high-quality novel ideas do appear in publication
> outlets that are universally recognized to be of higher quality than
> average.
>
> Thanks,
> Chitu
>
>
> -------- Message original --------
> Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or
> exclude conference articles (was Request to verify articles for Wikipedia
> literature review)
> De : Jack Park &lt;jackpark(a)gmail.com&gt;
> Pour : Research into Wikimedia content and communities
>     &lt;wiki-research-l(a)lists.wikimedia.org&gt;
> Date : 15/03/2011 5:26 PM
>
> When you consider a "top tier conference", how do you know you are not
> excluding contributions that might be not just novel but also truly
> important?
>
> It seems that page rank plays the role of beauty contest in the sense
> that top-ranked pages are those already in the view of others. I have
> seen comments that this filters against novelty, possibly crucial
> novelty.
>
> Jack
>
> On Tue, Mar 15, 2011 at 11:56 AM, Chitu Okoli &lt;Chitu.Okoli(a)concordia.ca&gt;
> wrote:
>
> We considered including top-tier conferences, but the question is, what is a
> "top conference"? In trying to answer this, we looked at a couple of
> sources:
...
  * Top Tier and 2nd tier conferences from
 http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
 * A-ranked conferences in Information and Computing Sciences from
 http://lamp.infosys.deakin.edu.au/era/?pageÏorsel10 > * We also considered
including all WikiSym articles on Wikipedia
>
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>

--
 :: Andrea Forte
 :: Assistant Professor
 :: College of Information Science and Technology, Drexel University
 :: http://www.andreaforte.net

-- 
 :: Andrea Forte
 :: Assistant Professor
 :: College of Information Science and Technology, Drexel University
 :: http://www.andreaforte.net

Wikipedia literature review - include or exclude conference articles