Hi Jack,

Actually, the reason we're talking about top-tier is based on the same reason we talk about peer-reviewed versus non-peer-reviewed. No one can argue that non-peer-reviewed work (such as working papers) often have completely novel ideas. The problem is that someone has to wade through tens of thousands of works of hugely varying quality to find a few pearls. The peer-review process does this wading; while it might miss a few novel items, it would probably get most of the high-quality ones. Similarly, there are at least 2,000 Wikipedia studies. Since we can't go through all of them, we hope that most of the high-quality novel ideas do appear in publication outlets that are universally recognized to be of higher quality than average.

Thanks,
Chitu


-------- Message original --------
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles (was Request to verify articles for Wikipedia literature review)
De : Jack Park <jackpark@gmail.com>
Pour : Research into Wikimedia content and communities
    <wiki-research-l@lists.wikimedia.org>
Date : 15/03/2011 5:26 PM
When you consider a "top tier conference", how do you know you are not
excluding contributions that might be not just novel but also truly
important?

It seems that page rank plays the role of beauty contest in the sense
that top-ranked pages are those already in the view of others. I have
seen comments that this filters against novelty, possibly crucial
novelty.

Jack

On Tue, Mar 15, 2011 at 11:56 AM, Chitu Okoli <Chitu.Okoli@concordia.ca> wrote:
We considered including top-tier conferences, but the question is, what is a
"top conference"? In trying to answer this, we looked at a couple of
sources:
* Top Tier and 2nd tier conferences from
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?pageĻorsel10
* We also considered including all WikiSym articles on Wikipedia