Wow my mailer must have tricked me into responding to the wrong piece of email. Sorry for
the unnecessary noise, folks.
-E
On Mar 16, 2011, at 1:34 PM, Eric Bloch wrote:
yay!
On Mar 16, 2011, at 1:08 PM, Chitu Okoli wrote:
Sorry for the late responses; with classes, meetings, office hours, baby, and so on, I
can't respond as fast as I'd like, but I'm really grateful for all the great
responses.
Thanks, James for the ideas you've suggested; I summarize them thus:
* Publication date cut-off: We'll play with these and see how many we're left
with.
* Randomize: ha ha ha
* Topic/empirical vs. conceptual/quantitative vs. qualitative: Actually, one of the
features of our review is that we explicitly want to include non-computer science works in
our review, many of which are conceptual, qualitative, and covering unusual topics (e.g.
music). Any of these criteria would systematically exclude these articles. Unfortunately,
we see that our journal vs. conference cut-off systematically excludes many computer
science articles :-(
* Cited articles: We hadn't thought about this; I'll talk more about it in
responding to Travis' thread.
* Adding more reviewers: I'll follow up on this in responding to Reid's thread.
Thanks a lot.
~ Chitu
-------- Message original --------
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude
conference articles (was Request to verify articles for Wikipedia literature review)
De : James Howison <james@howison.name><mailto:james@howison.name>
Pour : Research into Wikimedia content and communities
<wiki-research-l@lists.wikimedia.org><mailto:wiki-research-l@lists.wikimedia.org>
Date : 15/03/2011 4:57 PM
I am a little sheepish; clearly you've really struggled with this, it's certainly
a huge amount of papers.
I'm tempted to ask what happens if you cut by publication date, but I suspect that
that doesn't help much because of the accelerating rate of publication. In any case
not entirely sure of the justification for not including older things, it's not as
though one stops knowing them :)
Ah, I know: randomize ;) Ok, that's not really in the spirit of a review article.
Have you considered cutting by some first quick pass characteristics, such as topic (using
some framework relevant to your interests, we used Input-Process-Output for organizing
studies of FLOSS)/empirical vs conceptual/perhaps even quant. vs qual. That is, of
course, a lot of work just there but it seems to deal with the selection bias the best.
That would also help give a conceptual focus to the review article.
To avoid the full selection bias of excluding conferences, perhaps you could include only
those that are cited in your journal articles? (hmmm, issues there, but perhaps worth
thinking about; could one seek out some variant of "the connected set" of
articles, with some cutting factor on the strength of linkage to bring the number down to
something managable?).
Adding people to your review team is another option, I'm sure you've thought about
that. Difficulties there are obvious (a good review goes beyond 'tagging' articles
and conducts cross-cutting conceptually organized perspective, hard to coordinate or build
through disconnected work).
Best wishes for the work,
James
On Mar 15, 2011, at 14:56, Chitu Okoli wrote:
James and Travis, you bring up a point that we have struggled back and forth with for
several months. We really, really would like to include conference articles, but we just
can't see how we could handle many more articles than what we've got now.
We've been working on and off on this project for over two years now. (You can find
works in progress at the link at the bottom to my website.) We'd like to get it done
eventually, and we can only handle so many articles.
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l