Andrew Gray andrew.gray@dunelm.org.uk writes:
They worked on a journal basis, classing them as "OA" or "not OA". But this is, in some ways, a very small sample. See, eg/, pp. http://science-metrix.com/files/science-metrix/publications/d_1.8_sm_ec_dg-r..., which suggests that articles in gold OA titles represent less than 15% of the total amount "freely available" through various forms.
Given this limitation, it seems quite plausible that the actual OA:citation correlation is higher on a *per-paper* basis... we just don't really have the information to be sure.
I agree, though I think one reason the studies are oriented that way (besides convenience of data) is that a main policy-evaluation goal of such studies is to answer the question: is the OA / non-OA decision for a journal one that will significantly affect their impact factor? If going OA improves citations, then that's an argument in favor of going OA. To answer that question, the relevant comparison is OA vs. non-OA journals (preferably comparing similarly situated journals within fields).
Papers from non-OA journals that are nonetheless available on the internet for free make more a more complex case, both in terms of how they got there, and what the policy implications should be. Though still an interesting case!