Hello,
if I remember correctly there was some interesting research discussed here about citations to articles cited on Wikipedia. Can someone help me find that work?
It is hard to google for, because I keep turning up research on citing Wikipedia itself, which is not what I'm after. The question is: "Are articles that appear in Wikipedia citations preferentially cited when compared with other comparable articles?"
This is related to the so-called "Matthew Effect" and along these lines I found a recent article that says "controlling for field and impact factor, the odds that an open access journal is referenced on the English Wikipedia are 47% higher compared to closed access journals." http://arxiv.org/abs/1506.07608
However, I'm interested in the "downstream" side, not the "upstream" side.
TIA.
Joe
..."the odds that an open access journal is referenced on the English Wikipedia are 47% higher compared to closed access"
Thanks for posting! That's an interesting paper, for all sorts of reasons. I read it because I highly doubt that the number is as low as that. There is an increasing preference for open access external links just because it's becoming the only way for newbies to avoid the AfD queue. I was disappointed that the stats were not limited to external links, as I think that would be more interesting (the print references have often been grandfathered in, while the online references generally are more recently added by Wikipedians). I was really puzzled though by the graph comparing citations across various language-pedias (Galician at rank 7th !?!).
Also it would be interesting to know how many of those originally English citations are migrating into other wiki languages through the translation tool. I know I reuse citations from non-English languages on occasion when I can't find anything in English while creating non-existent articles in English from Wikidata info that have articles out there in other languages.
On Wed, Aug 19, 2015 at 12:05 PM, Joe Corneli holtzermann17@gmail.com wrote:
Hello,
if I remember correctly there was some interesting research discussed here about citations to articles cited on Wikipedia. Can someone help me find that work?
It is hard to google for, because I keep turning up research on citing Wikipedia itself, which is not what I'm after. The question is: "Are articles that appear in Wikipedia citations preferentially cited when compared with other comparable articles?"
This is related to the so-called "Matthew Effect" and along these lines I found a recent article that says "controlling for field and impact factor, the odds that an open access journal is referenced on the English Wikipedia are 47% higher compared to closed access journals." http://arxiv.org/abs/1506.07608
However, I'm interested in the "downstream" side, not the "upstream" side.
TIA.
Joe
-- RMS: "I am not on vacation, but I am at the end of a long time delay. I am located somewhere on Earth, but as far as responding to email is concerned, I appear to be well outside the solar system."
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On 20 August 2015 at 06:54, Jane Darnell jane023@gmail.com wrote:
..."the odds that an open access journal is referenced on the English Wikipedia are 47% higher compared to closed access"
Thanks for posting! That's an interesting paper, for all sorts of reasons. I read it because I highly doubt that the number is as low as that. There is
I've been meaning to actually go through this paper for a while, and finally did so this morning :-).
They worked on a journal basis, classing them as "OA" or "not OA". But this is, in some ways, a very small sample. See, eg/, pp. http://science-metrix.com/files/science-metrix/publications/d_1.8_sm_ec_dg-r..., which suggests that articles in gold OA titles represent less than 15% of the total amount "freely available" through various forms.
Given this limitation, it seems quite plausible that the actual OA:citation correlation is higher on a *per-paper* basis... we just don't really have the information to be sure.
Andrew Gray andrew.gray@dunelm.org.uk writes:
They worked on a journal basis, classing them as "OA" or "not OA". But this is, in some ways, a very small sample. See, eg/, pp. http://science-metrix.com/files/science-metrix/publications/d_1.8_sm_ec_dg-r..., which suggests that articles in gold OA titles represent less than 15% of the total amount "freely available" through various forms.
Given this limitation, it seems quite plausible that the actual OA:citation correlation is higher on a *per-paper* basis... we just don't really have the information to be sure.
I agree, though I think one reason the studies are oriented that way (besides convenience of data) is that a main policy-evaluation goal of such studies is to answer the question: is the OA / non-OA decision for a journal one that will significantly affect their impact factor? If going OA improves citations, then that's an argument in favor of going OA. To answer that question, the relevant comparison is OA vs. non-OA journals (preferably comparing similarly situated journals within fields).
Papers from non-OA journals that are nonetheless available on the internet for free make more a more complex case, both in terms of how they got there, and what the policy implications should be. Though still an interesting case!
Dear all, thanks for the great thread - what do you think of turning it into a contribution to the next issue of the Research Newsletter, which is likely to be started at https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2015-08-26/Recent... soon? During the Wikimania hackathon, we also discussed this paper, and we had an email exchange with the authors as well as the writer of a review of the paper: https://keet.wordpress.com/2015/07/12/wikipedia-open-access-not-quite-a-revo... . Will dig out my notes on it and try to contribute to the Signpost piece.
Cheers, Daniel
On Thu, Aug 20, 2015 at 9:15 PM, Mark J. Nelson mjn@anadrome.org wrote:
Andrew Gray andrew.gray@dunelm.org.uk writes:
They worked on a journal basis, classing them as "OA" or "not OA". But this is, in some ways, a very small sample. See, eg/, pp. http://science-metrix.com/files/science-metrix/publications/d_1.8_sm_ec_dg-r..., which suggests that articles in gold OA titles represent less than 15% of the total amount "freely available" through various forms.
Given this limitation, it seems quite plausible that the actual OA:citation correlation is higher on a *per-paper* basis... we just don't really have the information to be sure.
I agree, though I think one reason the studies are oriented that way (besides convenience of data) is that a main policy-evaluation goal of such studies is to answer the question: is the OA / non-OA decision for a journal one that will significantly affect their impact factor? If going OA improves citations, then that's an argument in favor of going OA. To answer that question, the relevant comparison is OA vs. non-OA journals (preferably comparing similarly situated journals within fields).
Papers from non-OA journals that are nonetheless available on the internet for free make more a more complex case, both in terms of how they got there, and what the policy implications should be. Though still an interesting case!
-- Mark J. Nelson Anadrome Research http://www.kmjn.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Regarding the question from the OP [oh, that's me!], here are some search terms that produce relevant results,
citation frequency wikipedia -"cite wikipedia" -"citing sources"
Interestingly, the first paper I found suggests that the trend is the almost the opposite of the one I was thinking would obtain.
«In the present study, we show that inclusion of articles in Wikipedia does not increase the propensity of articles to be cited. Interestingly, the reverse is reported to be true, i.e., Wikipedia selectively lists high impact articles shortly after their publication.»
- Marashi et al., "Impact of Wikipedia on citation trends"
This work is mentioned in a previous Research Newsletter, here:
https://meta.wikimedia.org/wiki/Research:Newsletter/2013/August#Briefly
... which is probably what I was remembering.
A slightly expanded treatment by the same authors is available as:
"Impact of inclusion of scholarly references in Wikipedia on citation trends"
I don't see that the result has been replicated by other researchers.
Some related themes are discussed from an (optimistic) technical engineering point of view in, here:
«Our results show that a) classical query expansion using terms extracted from query pages leads to increased precision, and b) link distance between query and book pages in Wikipedia provides a good indicator of relevance that can boost the retrieval score of relevant books in the result ranking of a book search engine.»
- Koolen et al., "Wikipedia pages as entry points for book search"
-Joe
On Thu, Aug 20 2015, Daniel Mietchen wrote:
Dear all, thanks for the great thread - what do you think of turning it into a contribution to the next issue of the Research Newsletter, which is likely to be started at https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2015-08-26/Recent... soon? During the Wikimania hackathon, we also discussed this paper, and we had an email exchange with the authors as well as the writer of a review of the paper: https://keet.wordpress.com/2015/07/12/wikipedia-open-access-not-quite-a-revo... . Will dig out my notes on it and try to contribute to the Signpost piece.
Cheers, Daniel
On Thu, Aug 20, 2015 at 9:15 PM, Mark J. Nelson mjn@anadrome.org wrote:
Andrew Gray andrew.gray@dunelm.org.uk writes:
They worked on a journal basis, classing them as "OA" or "not OA". But this is, in some ways, a very small sample. See, eg/, pp. http://science-metrix.com/files/science-metrix/publications/d_1.8_sm_ec_dg-r..., which suggests that articles in gold OA titles represent less than 15% of the total amount "freely available" through various forms.
Given this limitation, it seems quite plausible that the actual OA:citation correlation is higher on a *per-paper* basis... we just don't really have the information to be sure.
I agree, though I think one reason the studies are oriented that way (besides convenience of data) is that a main policy-evaluation goal of such studies is to answer the question: is the OA / non-OA decision for a journal one that will significantly affect their impact factor? If going OA improves citations, then that's an argument in favor of going OA. To answer that question, the relevant comparison is OA vs. non-OA journals (preferably comparing similarly situated journals within fields).
Papers from non-OA journals that are nonetheless available on the internet for free make more a more complex case, both in terms of how they got there, and what the policy implications should be. Though still an interesting case!
-- Mark J. Nelson Anadrome Research http://www.kmjn.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Andrew Gray, 20/08/2015 14:21:
They worked on a journal basis, classing them as "OA" or "not OA".
Weird, why didn't they just use DOAJ? https://doaj.org/
Nemo
They did; DOAJ seems to have been the method used to determine whether a journal was OA or not (which is fair enough).
Andrew.
On 21 August 2015 at 12:50, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Andrew Gray, 20/08/2015 14:21:
They worked on a journal basis, classing them as "OA" or "not OA".
Weird, why didn't they just use DOAJ? https://doaj.org/
Nemo
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
wiki-research-l@lists.wikimedia.org