Xmldatadumps-l January 2025

xmldatadumps-l@lists.wikimedia.org

5 participants
4 discussions

2025-01-22 Dumps failed delivery January dumps (enwiki, wikidatawiki)
by ahoelzl＠wikimedia.org 31 Jan '25

31 Jan '25

Greetings XML Dump users, TL;DR: This is a note to inform you that enwiki and wikidatawiki dumps have not been produced for January 2025. enwiki Dumps were explicitly paused due to causing issues with their production interference. The wikidatawiki dump has failed to run successfully at the beginning of January. We are working re-running the enwiki January data and resuming full dump operations as of February 1st. Longer: Enwiki and wikidatawiki dumps have not been produced for January 2025 https://dumps.wikimedia.org/enwiki/20250103/ https://dumps.wikimedia.org/wikidatawiki/20241201/ The latest respective dump versions are of December 2024. Respective dumps are skipped in the current partial run, see https://dumps.wikimedia.org/backup-index.html Enwiki data has been explicitly paused on December 22nd to not interfere with production wiki instances: https://phabricator.wikimedia.org/T368098#10421298 https://gerrit.wikimedia.org/r/c/operations/puppet/+/1106019 While the failed processing of wikidatawiki in January is yet not clear: https://phabricator.wikimedia.org/T383568 On January 6th we started working on switching dumps from production MariaDB to the analytics replicas (dbstore100[7-9]) to resolve the production dependency: https://phabricator.wikimedia.org/T382947 Patches for the change were deployed on Jan 20th https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1109109 An initial test was successfully run on the eswiki. We are further working on switching all wikis over to the analytics replicas: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1113475 Recommendation: Pause dependent pipeline until data will become available. This should be the case for the February dump runs.

1 2

XML Dumps FAQ monthly update
by noreply.xmldatadumps＠wikimedia.org 25 Jan '25

25 Jan '25

Greetings XML Dump users and contributors! This is your automatic monthly Dumps FAQ update email. This update contains figures for the 20241201 full revision history content run. We are currently dumping 1010 projects in total. --------------------- Stats for amwikiquote on date 20250101 Total size of page content dump files for articles, current content only: 361,850 Total size of page content dump files for all pages, current content only: 856,843 Total size of page content dump files for all pages, all revisions: 1,945,415 --------------------- Stats for enwiki on date 20241201 Total size of page content dump files for articles, current content only: 105,373,313,207 Total size of page content dump files for all pages, current content only: 216,495,087,582 Total size of page content dump files for all pages, all revisions: No info available (no files found) --------------------- Sincerely, Your friendly Wikimedia Dump Info Collector

2 1

Temporary pause of enwiki dumps until the New Year
by Ben Tullis 21 Jan '25

21 Jan '25

Hello XML dumps users and contributors, Unfortunately, we have had to interrupt the enwiki dump for 20241220 <https://dumps.wikimedia.org/enwiki/20241220/> and delay the start of the enwiki dump on January 1st. This is due to some performance issues in the database layer, which have been identified in https://phabricator.wikimedia.org/T368098 and have been causing significant impact to on-wiki workflows, as well as out-of-hours notifications to our SRE teams. We do not do this lightly and we apologise for any inconvenience that this may cause you, as we understand how important these data sets are. Please be assured that we at the WMF are working hard on improving the dumps architecture at the moment, on multiple fronts. We will follow up with further information early in the New Year, when we re-enable the enwiki dump. Kind regards, Ben -- *Ben Tullis*(he/him) Staff Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/>

2 1

Proposal to sunset "Page abstract for Yahoo" dumps
by Amir Sarabadani 02 Jan '25

02 Jan '25

Hi, I'm proposing to sunset "Extracted page abstracts for Yahoo" dumps. We have to maintain a dedicated extension called ActiveAbstract <https://www.mediawiki.org/wiki/Extension:ActiveAbstract> just to produce this type of dumps while It's rather easy to extract the same values from the main dump. There are also other reasons outlined in the ticket: T382069 <https://phabricator.wikimedia.org/T382069> If you rely on this type of dumps and can't easily switch or if I missed something obvious about page abstract dumps or ActiveAbstract extension, please make a comment in the ticket. Thank you -- Amir (he/him)

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Xmldatadumps-l January 2025