I'm happy to announce a new mirror for datasets other than the XML dumps. This mirror comes to us courtesy of the Center for Research Computing, University of Notre Dame, and covers everything "other" [1] which includes such goodies as Wikidata entity dumps, pageview counts, titles of all files on each wiki (daily), titles of all articles of each wiki (daily), and the so-called "adds-changes" dumps, among other things. You can access it at http://wikimedia.crc.nd.edu/other/ so please do!
Ariel
Ariel Glenn WMF, 04/05/2016 14:33:
You can access it at http://wikimedia.crc.nd.edu/other/ so please do!
Great news, especially because it's ten times faster than dumps.wikimedia.org! Finally, every time I need a dataset to quickly verify a sudden idea I have, the download becomes a matter of minutes rather than hours.
Nemo
Dear all,
The server hosting this service has been moved to a different network, and as such, it is now "only accessible/routable from select (still many) members of Internet2 (U.S. universities), ESnet (U.S. national labs), and Geant in Europe. This restricted list of places is currently limited, but is continually growing", as email from our contact at that mirror says. For folks from specific institutions that suddenly no longer have access, I can forward instution names along and hope that helps.
Ariel
On Wed, May 4, 2016 at 3:33 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
I'm happy to announce a new mirror for datasets other than the XML dumps. This mirror comes to us courtesy of the Center for Research Computing, University of Notre Dame, and covers everything "other" [1] which includes such goodies as Wikidata entity dumps, pageview counts, titles of all files on each wiki (daily), titles of all articles of each wiki (daily), and the so-called "adds-changes" dumps, among other things. You can access it at http://wikimedia.crc.nd.edu/other/ so please do!
Ariel
Ariel Glenn WMF, 17/06/2016 13:21:
For folks from specific institutions that suddenly no longer have access, I can forward instution names along and hope that helps.
It would be nice to whitelist the wmflabs.org servers, which would benefit from a faster server to download this stuff from.
Nemo
Federico Leva (Nemo), 17/06/2016 14:59:
Ariel Glenn WMF, 17/06/2016 13:21:
For folks from specific institutions that suddenly no longer have access, I can forward instution names along and hope that helps.
It would be nice to whitelist the wmflabs.org servers, which would benefit from a faster server to download this stuff from.
Did this prove impossible? I need mediacounts data on a Labs server now, and it would take days do download from dumps.wikimedia.org.
Nemo
I got nothing back from my email so I assume that means it's not happening.
http://dumps.wikimedia.your.org/other/mediacounts/daily/2016/ There are mediacounts here, is the download speed acceptable?
Ariel
On Tue, Sep 27, 2016 at 12:34 PM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Federico Leva (Nemo), 17/06/2016 14:59:
Ariel Glenn WMF, 17/06/2016 13:21:
For folks from specific institutions that suddenly no longer have access, I can forward instution names along and hope that helps.
It would be nice to whitelist the wmflabs.org servers, which would benefit from a faster server to download this stuff from.
Did this prove impossible? I need mediacounts data on a Labs server now, and it would take days do download from dumps.wikimedia.org.
Nemo
Xmldatadumps-l mailing list Xmldatadumps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
Ok.
Ariel Glenn WMF, 27/09/2016 11:47:
http://dumps.wikimedia.your.org/other/mediacounts/daily/2016/ There are mediacounts here, is the download speed acceptable?
Oh yes, that's around 50 MiB/s. I did not see this directory linked from their main page so I thought they had removed it; I'll add the link from https://meta.wikimedia.org/wiki/Mirroring_Wikimedia_project_XML_dumps
Nemo
Thanks, that's great.
Ariel
On Tue, Sep 27, 2016 at 1:13 PM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Ok.
Ariel Glenn WMF, 27/09/2016 11:47:
http://dumps.wikimedia.your.org/other/mediacounts/daily/2016/ There are mediacounts here, is the download speed acceptable?
Oh yes, that's around 50 MiB/s. I did not see this directory linked from their main page so I thought they had removed it; I'll add the link from https://meta.wikimedia.org/wiki/Mirroring_Wikimedia_project_XML_dumps
Nemo
xmldatadumps-l@lists.wikimedia.org