Xmldatadumps-l January 2017

xmldatadumps-l@lists.wikimedia.org

5 participants
5 discussions

revised index.html for dumps?
by Ariel Glenn WMF 31 Jan '17

31 Jan '17

Hey folks, A kind person submitted a patch to make the index.html page, and potentially others as well, nicer. Have a look: https://phabricator.wikimedia.org/T155697 and please comment there if you have suggestions. Feel free to forward this to anyone else who might be interested. Thanks! Ariel

2 2

Fwd: Divide XML dumps by page.page_namespace (and figure out what to do with the "pages-articles" dump)
by Federico Leva (Nemo) 18 Jan '17

18 Jan '17

Input requested: https://lists.wikimedia.org/pipermail/wikitech-l/2017-January/087393.html , https://phabricator.wikimedia.org/T99483 Personally I think that the main issue is the slowness of some of the tools people use (including dumps.wikimedia.org itself), so I tried to improve the docs a bit with what I learnt: * https://meta.wikimedia.org/wiki/Data_dumps/Download_tools#Downloading_the_X… * https://meta.wikimedia.org/wiki/Data_dumps#Faster_archives_and_servers Nemo P.s.: In the last few weeks I archived 20+ TiB of Wikimedia Commons files on Internet Archive and I'm continuing, see http://archiveteam.org/index.php?title=Wikimedia_Commons .

1 0

dumps not making it to labs in timely manner
by Bryan White 06 Jan '17

06 Jan '17

The last two two cycles, the dumps have not been moved to Labs as the individual files complete. On the 20171220 cycle, the files were not transferred until after the 20170101 cycle had started. The 20170101 files are not showing up on Labs. Bryan

2 1

Re: [Xmldatadumps-l] Xmldatadumps-l Digest, Vol 81, Issue 1
by Cameron Ehrlich 03 Jan '17

03 Jan '17

Unsubscribe On Mon, Jan 2, 2017 at 4:00 AM <xmldatadumps-l-request(a)lists.wikimedia.org> wrote: > Send Xmldatadumps-l mailing list submissions to > xmldatadumps-l(a)lists.wikimedia.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l > or, via email, send a message with subject or body 'help' to > xmldatadumps-l-request(a)lists.wikimedia.org > > You can reach the person managing the list at > xmldatadumps-l-owner(a)lists.wikimedia.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Xmldatadumps-l digest..." > > > Today's Topics: > > 1. Having trouble with November 2016 enwiki (Phil Hunt) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 2 Jan 2017 03:43:04 +0000 > From: Phil Hunt <cabalamat(a)gmail.com> > To: xmldatadumps-l(a)lists.wikimedia.org > Subject: [Xmldatadumps-l] Having trouble with November 2016 enwiki > Message-ID: > < > CANrmF702ZtJ+MU4yTcbnGAXr4mJdXx8hrUsFs+5CJg+6ffnqQQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, > > I have a local mediawiki installation and I'm trying to import the Nov 2016 > en-wikipedia into it > (enwiki-20161101-pages-articles-multistream.xml.bz) > > > I'm using importDump.php, but the commands I try cause it to immediately > fall over with an error message. When I do: > > $ php importDump.php < > ~/data/wikifork/enwiki-20161101-pages-articles-multistream.xml.bz2 > I get: > > PHP Warning: XMLReader::read(): > uploadsource://405216067937ad302ddd05631e3ee446:1: parser error : Document > is empty in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 538 > > PHP Warning: XMLReader::read(): BZh91AY&SY귟C in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 538 > > PHP Warning: XMLReader::read(): ^ in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 538 > > [651d62f6f956c1552494dab7] [no req] MWException from line 542 of > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php: > Expected <mediawiki> tag, got > > Backtrace: > > #0 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(318): > WikiImporter->doImport() > > #1 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(277): > BackupReader->importFromHandle(resource) > > #2 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(108): > BackupReader->importFromStdin() > > #3 /home/phil/sproj/wikifork/mediawiki/maintenance/doMaintenance.php(111): > BackupReader->execute() > > #4 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(323): > require_once(string) > > #5 {main} > > And when I do: > > $ php importDump.php --conf ../LocalSettings.php > ~/data/wikifork/enwiki-20161101-pages-articles-multistream.xml.bz2 my_wiki > > I get: > > > PHP Warning: XMLReader::read(): > uploadsource://cd1f7162d36eda06a29d64becaf0b825:45: parser error : Extra > content at the end of the document in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 512 > > PHP Warning: XMLReader::read(): </siteinfo> in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 512 > > PHP Warning: XMLReader::read(): ^ in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 512 > > PHP Warning: XMLReader::read(): Load Data before trying to read in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 603 > > PHP Warning: XMLReader::read(): Load Data before trying to read in > /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on > line 578 > > Done! > > You might want to run rebuildrecentchanges.php to regenerate RecentChanges > > > I'm puzzled that I'm getting different error messages here. > > Am I going down the wrong track entirely and need to use mwdumper? > > > -- > Phil Hunt, <cabalamat(a)gmail.com> >

1 0

Having trouble with November 2016 enwiki
by Phil Hunt 02 Jan '17

02 Jan '17

Hi, I have a local mediawiki installation and I'm trying to import the Nov 2016 en-wikipedia into it (enwiki-20161101-pages-articles-multistream.xml.bz) I'm using importDump.php, but the commands I try cause it to immediately fall over with an error message. When I do: $ php importDump.php < ~/data/wikifork/enwiki-20161101-pages-articles-multistream.xml.bz2 I get: PHP Warning: XMLReader::read(): uploadsource://405216067937ad302ddd05631e3ee446:1: parser error : Document is empty in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 538 PHP Warning: XMLReader::read(): BZh91AY&SY귟C in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 538 PHP Warning: XMLReader::read(): ^ in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 538 [651d62f6f956c1552494dab7] [no req] MWException from line 542 of /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php: Expected <mediawiki> tag, got Backtrace: #0 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(318): WikiImporter->doImport() #1 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(277): BackupReader->importFromHandle(resource) #2 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(108): BackupReader->importFromStdin() #3 /home/phil/sproj/wikifork/mediawiki/maintenance/doMaintenance.php(111): BackupReader->execute() #4 /home/phil/sproj/wikifork/mediawiki/maintenance/importDump.php(323): require_once(string) #5 {main} And when I do: $ php importDump.php --conf ../LocalSettings.php ~/data/wikifork/enwiki-20161101-pages-articles-multistream.xml.bz2 my_wiki I get: PHP Warning: XMLReader::read(): uploadsource://cd1f7162d36eda06a29d64becaf0b825:45: parser error : Extra content at the end of the document in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 512 PHP Warning: XMLReader::read(): </siteinfo> in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 512 PHP Warning: XMLReader::read(): ^ in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 512 PHP Warning: XMLReader::read(): Load Data before trying to read in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 603 PHP Warning: XMLReader::read(): Load Data before trying to read in /home/phil/sproj/wikifork/mediawiki/includes/import/WikiImporter.php on line 578 Done! You might want to run rebuildrecentchanges.php to regenerate RecentChanges I'm puzzled that I'm getting different error messages here. Am I going down the wrong track entirely and need to use mwdumper? -- Phil Hunt, <cabalamat(a)gmail.com>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Xmldatadumps-l January 2017