On 3/29/2010 6:08 PM, River Tarnell wrote: I use the pageview data currently, for similar reasons that Lars mentioned. I produce monthly pageview data reports for Wikiprojects on enwiki.[1] Currently I only keep the data for the projects that are subscribed to the service (305 projects, ~2.3 million pages for April 2010) If it were in the database, it could be easier, though I'd have to rewrite a lot of stuff. I only need monthly data for what I'm currently doing, but daily data could potentially be useful for other things. The current (raw) format is fine for me, though it would be nice if the files used a more consistent naming. Most are in the form: pagecounts-YYYYMMDD-HH0000.gz but every once in a while, pagecounts-YYYYMMDD-HH0001.gz and very rarely, things like pagecounts-YYYYMMDD-HH2001.gz and a few other variations. Also, every once in a while, there are long delays before files are available or files are missing entirely. If there is interest in having this in the database somewhere, I might be able to help out in terms of coding, as much of what I have could be fairly easily adapted to extract the data for all pages on all projects, rather than just the ones on the English Wikipedia that my tool needs. [1] http://toolserver.org/~alexz/pop/view.php -- Alex (wikipedia:en:User:Mr.Z-man)

Hi, Am 30.03.2010, 01:08 Uhr, schrieb River Tarnell <river.tarnell(a)wikimedia.de>de>: I think there are some tools, which could use this data as "how popular is an article" for sorting results. For this monthly data would be fine. Daily data is great, too - maybe the sum of the last 30 days or so could be cached somewhere. More detailed data (like hourly) seems too precise for me, but of course one could imagine even tools for per-minute data: e.g. the usage stats for articles relevant for a question at "who wants to be a millionaire" would be very interesting. But there are only few such things, so this could be done on raw data, if needed. So I would propose (if possible) daily data with a db view for either "last month" or better "last 30 days" (I don't know, if something like this could be cached... or maybe a cron for updating extra tables...) Sincerelly, Christian Thiele / user:apper

River Tarnell wrote: Being this user, I can only be glad if this happens :-) And I'll be glad to help setting this up if I can. Are you thinking of storing the raw data (something desperately needed, even though there are a few copies around), or just the formatted data ? It seems that many people have been working with this data in completely independent way; Erik Zachte has a set of scripts to reformat it (which I have used), but several other people have developed their own tools. It'd be nice to have a common format (better that the current "raw" data). The interest for this goes beyond the toolserver, so I have just created a (very short, for now) page on meta.wikipedia.org to store information related to these files: http://meta.wikimedia.org/wiki/Statistics/Consultation If this information is already available elsewhere and I missed it, please let me know. Frédéric

Platonides wrote: Common API would be nice, but that means having a service able to provide the data. At the moment, it is already quite difficult for people to access the files containing the raw data -- well, it is not *so* difficult, but there is no central location containing all the data, and the files are cumbersome to read and process (as already mentioned here). So in the short run, we need an easy way to provide access to the data files. Anything above that will be icing on the cake. Frédéric