Xmldatadumps-l

xmldatadumps-l@lists.wikimedia.org

1 participants
728 discussions

Dump for enwiki blocked ?
by Nicolas Vervelle 22 Oct '18

22 Oct '18

Hi, The dump for enwiki seems to be blocked in "First-pass for page XML data dumps" : 2018-10-20 10:49:29 in-progress First-pass for page XML data dumps 2018-10-22 09:32:02: enwiki (ID 298305) 416 pages (0.8|30.1/sec all|curr), 36000 revs (70.5|72.4/sec all|curr), ETA 2019-03-13 10:20:01 [max 865183738] Other dumps like frwiki seem blocked also, maybe waiting for enwiki ? Nico

2 1

some revisions missing from Sept 13 adds-changes dump
by Ariel Glenn WMF 12 Oct '18

12 Oct '18

If you are a user of the adds-changes (so-called "incremental") dumps, read on. All dumps use database servers in our eqiad data center. For the past month, the wiki projects have used primary database masters out of our codfw data center; on one of these days, a number of revisions did not replicate properly to eqiad for about 50 minutes. This was not discovered until we switched back to using the database servers in eqiad as primary masters. The date that replication was broken was September 13th. Because dumps use eqiad database servers, the adds-changes dumps published on that date are also missing that data. We suggest that if you are using adds-changes dumps for a local mirror, you import the next regular run of xml/sql dumps, starting on Oct 20th (for current revisons only) or Nov 1st (for full history), which should contain the missing revisions. For more information on this incident, you may follow: https://phabricator.wikimedia.org/T206743 Ariel

1 0

flow dumps failures being worked on
by Ariel Glenn WMF 05 Oct '18

05 Oct '18

Hello dumps users! You may have noticed that a number of wikis have had dumps failures on the flow dumps step. The cause is known (a cleanup of mediawiki core that didn't carry over to the extension) and these jobs should be fixed up today or tomorrow. Ariel

1 2

adds-changes (so-called 'incremental') dumps failed today
by Ariel Glenn WMF 03 Oct '18

03 Oct '18

The failure was a side effect of a configuration change that will, ironically enough, make it easier to test the 'other' dumps, including eventually these ones, in mediawiki-vagrant; see https://phabricator.wikimedia.org/T201478 for more information about that. They should run tomorrow and contain the content for the missing run as well. Apologies for the inconvenience. Ariel

1 1

Oct 3 2018: RFC on xml dumps schema update to be discussed at TechCom
by Ariel Glenn WMF 01 Oct '18

01 Oct '18

Hey dumps users and contributors! This Wednesday, Oct 3 at 2pm PST(21:00 UTC, 23:00 CET) in #wikimedia-office TechCom will have a discussion about the RFC for the upcomign xml schema update needed for Multi-Content Revision content. Phabricator task: https://phabricator.wikimedia.org/T199121 TechCom minutes announcing the meeting: https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes/2018-0… Draft RFC: https://www.mediawiki.org/wiki/Requests_for_comment/Schema_update_for_multi… If you have comments or suggestions, please show up! Thanks, Ariel

1 0

XML Dumps FAQ monthly update
by noreply.xmldatadumps＠wikimedia.org 01 Oct '18

01 Oct '18

Greetings XML Dump users and contributors! This is your automatic monthly Dumps FAQ update email. This update contains figures for the 20180901 full revision history content run. We are currently dumping 912 projects in total. --------------------- Stats for pamwiki on date 20180901 Total size of page content dump files for articles, current content only: 41777521 Total size of page content dump files for all pages, current content only: 44732421 Total size of page content dump files for all pages, all revisions: 1314983462 --------------------- Stats for enwiki on date 20180901 Total size of page content dump files for articles, current content only: 68715508465 Total size of page content dump files for all pages, current content only: 153659978710 Total size of page content dump files for all pages, all revisions: 17959415517241 --------------------- Sincerely, Your friendly Wikimedia Dump Info Collector

1 0

Question
by Uki Susa 30 Sep '18

30 Sep '18

For What?

2 2

Current size of English Wikipedia Dump
by Andy Famiglietti 27 Sep '18

27 Sep '18

Hi All. Does anyone have an estimate of the current size of the full (all pages, all revisions) dump for the english Wikipedia? My understanding was that the full dump was about 10 terabytes as of May 2016, but when I grabbed one of the 7zip files and decompressed, it came to a full 72 gigabytes. That, times the 500 odd 7zip files would seem to indicate a 30TB dump or more! Does anyone know the actual size of the decompressed dump? - Andy Famiglietti New Media Scholar, Intermittent Wikipedian, Vaguely Humanoid

2 1

XML Dumps FAQ monthly update
by noreply.xmldatadumps＠wikimedia.org 26 Aug '18

26 Aug '18

Greetings XML Dump users and contributors! This is your automatic monthly Dumps FAQ update email. This update contains figures for the 20180801 full revision history content run. We are currently dumping 911 projects in total. --------------------- Stats for nowikiquote on date 20180801 Total size of page content dump files for articles, current content only: 3822926 Total size of page content dump files for all pages, current content only: 4981358 Total size of page content dump files for all pages, all revisions: 120506806 --------------------- Stats for enwiki on date 20180801 Total size of page content dump files for articles, current content only: 68301281689 Total size of page content dump files for all pages, current content only: 152796035982 Total size of page content dump files for all pages, all revisions: No info available (no files found) --------------------- Sincerely, Your friendly Wikimedia Dump Info Collector

1 0

huwiki, arwiki to be treated as 'big wikis' and run parallel jobs
by Ariel Glenn WMF 20 Aug '18

20 Aug '18

Starting September 1, huwiki and arwiki, which both take several days to complete the revsion history content dumps, will be moved to the 'big wikis' list, meaning that they will run jobs in parallel as do frwiki, ptwiki and others now, for a speedup. Please update your scripts accordingly. Thanks! Task for this: https://phabricator.wikimedia.org/T202268 Ariel

1 0

← Newer
1
...
19
20
21
22
23
24
25
...
73
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Xmldatadumps-l