Xmldatadumps-l

xmldatadumps-l@lists.wikimedia.org

720 discussions

XML Dumps FAQ monthly update
by noreply.xmldatadumps＠wikimedia.org 26 Aug '18

26 Aug '18

Greetings XML Dump users and contributors! This is your automatic monthly Dumps FAQ update email. This update contains figures for the 20180801 full revision history content run. We are currently dumping 911 projects in total. --------------------- Stats for nowikiquote on date 20180801 Total size of page content dump files for articles, current content only: 3822926 Total size of page content dump files for all pages, current content only: 4981358 Total size of page content dump files for all pages, all revisions: 120506806 --------------------- Stats for enwiki on date 20180801 Total size of page content dump files for articles, current content only: 68301281689 Total size of page content dump files for all pages, current content only: 152796035982 Total size of page content dump files for all pages, all revisions: No info available (no files found) --------------------- Sincerely, Your friendly Wikimedia Dump Info Collector

1 0

huwiki, arwiki to be treated as 'big wikis' and run parallel jobs
by Ariel Glenn WMF 20 Aug '18

20 Aug '18

Starting September 1, huwiki and arwiki, which both take several days to complete the revsion history content dumps, will be moved to the 'big wikis' list, meaning that they will run jobs in parallel as do frwiki, ptwiki and others now, for a speedup. Please update your scripts accordingly. Thanks! Task for this: https://phabricator.wikimedia.org/T202268 Ariel

1 0

missing adds-changes dumps, page titles for today
by Ariel Glenn WMF 08 Aug '18

08 Aug '18

These jobs did not run today due to a change in how maintenance scripts handle unknown arguments. The problem has been fixed and the jobs should run regularly tomorrow.

1 0

MultiContent Revisions and changes to the XML dumps
by Ariel Glenn WMF 02 Aug '18

02 Aug '18

As many of you may know, MultiContent Revisions are coming soon (October?) to a wiki near you. This means that we need changes to the XML dumps schema; these changes will likely NOT be backwards compatible. Initial discussion will take place here: https://phabricator.wikimedia.org/T199121 For background on MultiContent Revisions and their use on e.g. Commons or WikiData, see: https://phabricator.wikimedia.org/T200903 (Commons media medata) https://phabricator.wikimedia.org/T194729 (Wikidata entites) https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions (MCR generally) There may be other, better tickets/pages for background; feel free to supplement this list if you have such links. Ariel

1 0

XML Dumps FAQ monthly update
by noreply.xmldatadumps＠wikimedia.org 26 Jul '18

26 Jul '18

Greetings XML Dump users and contributors! This is your automatic monthly Dumps FAQ update email. This update contains figures for the 20180701 full revision history content run. We are currently dumping 909 projects in total. --------------------- Stats for idwikimedia on date 20180701 Total size of page content dump files for articles, current content only: 46311585 Total size of page content dump files for all pages, current content only: 46840000 Total size of page content dump files for all pages, all revisions: 998067435 --------------------- Stats for enwiki on date 20180701 Total size of page content dump files for articles, current content only: 67866278643 Total size of page content dump files for all pages, current content only: 151951530251 Total size of page content dump files for all pages, all revisions: No info available (no files found) --------------------- Sincerely, Your friendly Wikimedia Dump Info Collector

1 0

hewiki dump to be added to 'big wikis' and run with multiple processes
by Ariel Glenn WMF 20 Jul '18

20 Jul '18

Good morning! The pages-meta-history dumps for hewiki take 70 hours these days, the longest of any wiki not already running with parallel jobs. I plan to add it to the list of 'big wikis' starting August 1st, meaning that 6 jobs will run in parallel producing the usual numbered file output; look at e.g. frwiki dumps for an example. Please adjust any download/processing scripts accordingly. Thanks! Ariel

1 0

dumps.wikimedia.org web and rsync services down
by Bryan Davis 28 Jun '18

28 Jun '18

The https://dumps.wikimedia.org web interface for downloading various dump files is currently offline. The rsync service for external mirroring is as well. Local network NFS consumers may or may not be working depending on which server the consumer is attached to. This unexpected outage is the result of hardware issues following a short planned maintenance. We are currently investigating the root cause of the outage and will post additional updates as they become available. Thanks for your patience. Bryan -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA irc: bd808 v:415.839.6885 x6855

1 1

Re: [Xmldatadumps-l] [Wikitech-ambassadors] Queries regarding Wikimedia's Data Dumps.
by Nick Wilson (Quiddity) 07 Jun '18

07 Jun '18

(Fwding to Xmldatadumps-l@ list - https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l , CCing wikitech-ambassadors@ just for resolution here) On Mon, Jun 4, 2018 at 10:55 PM, Nikhil Prakash <nikhil07prakash(a)gmail.com> wrote: > Hi there, > > This is Nikhil, an undergraduate student from India. And I'm trying to > understand the Wikipedia's data dumps provided by Wikimedia. > > I'm working on 20180520 dumps. It contains many sections, each having > different data. And I would like to know what each section's data > represent. Although it's written in a brief, I don't get it clearly. > > Like in section "All pages, current versions only." Does each and every > article's current version is present in this data? Because I just > downloaded "enwiki-20180520-pages-meta-current1.xml-p10p30303.bz2", the > first-page information is of "AccessibleComputing" but it does not have > complete article's information in it? > > Hoping to get a quick reply. > > Thanks. > Nikhil > > > _______________________________________________ > Wikitech-ambassadors mailing list > Wikitech-ambassadors(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors > > -- Nick Wilson (Quiddity) Community Liaison, Wikimedia Foundation

1 0

Puthl
by teto00297 02 Jun '18

02 Jun '18

Sent from my Samsung Galaxy , an AT&T LTE smartphone

1 0

Re: [Xmldatadumps-l] Xmldatadumps-l Digest, Vol 94, Issue 1
by Astra Habibi 31 May '18

31 May '18

Pada tanggal 31 Mei 2018 7.00 PM, < xmldatadumps-l-request(a)lists.wikimedia.org> menulis: > > Send Xmldatadumps-l mailing list submissions to > xmldatadumps-l(a)lists.wikimedia.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l > or, via email, send a message with subject or body 'help' to > xmldatadumps-l-request(a)lists.wikimedia.org > > You can reach the person managing the list at > xmldatadumps-l-owner(a)lists.wikimedia.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Xmldatadumps-l digest..." > > > Today's Topics: > > 1. change to output file numbering of big wikis (Ariel Glenn WMF) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 31 May 2018 14:36:51 +0300 > From: Ariel Glenn WMF <ariel(a)wikimedia.org> > To: Wikipedia Xmldatadumps-l <Xmldatadumps-l(a)lists.wikimedia.org>, > Wikimedia developers <wikitech-l(a)lists.wikimedia.org> > Subject: [Xmldatadumps-l] change to output file numbering of big wikis > Message-ID: > < CALCvg_4Qmi2WsfiTFMqzHQBuX0fF0WZHhkz-YSWS7W0_zuBvZg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > TL;DR: > Scripts that reply on xml files numbered 1 through 4 should be updated to > check for 1 through 6. > > Explanation: > > A number of wikis have stubs and page content files generated 4 parts at a > time, with the appropriate number added to the filename. I'm going to be > increasing that thi month to 6. > > The reason for the increase is that near the end of the run there are > usually just a few big wikis taking their time at completing. If they run > with 6 processes at once, they'll finish up a bit sooner. > > If you have scripts that rely on the number 4, just increase it to 6 and > you're done. > > This will go into effect for the June 1 run and all runs afterwards. > > Thanks! >

1 0

← Newer
1
...
19
20
21
22
23
24
25
...
72
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Xmldatadumps-l