Xmldatadumps-l

xmldatadumps-l@lists.wikimedia.org

720 discussions

Strange order in the database dump progress
by Nicolas Vervelle 21 Mar '11

21 Mar '11

Hi, The order in the database dump progress<http://dumps.wikimedia.org/backup-index.html> page is quite strange. Some dumps in progress are not where they should, but even some complete dumps are not sorted (for 03/19 : up to 08:40 then from 05:27 to 00:33, with events from 08:38 to 08:27, or 00:13 later in the page) Nico - 2011-03-19 14:59:21 plwikinews<http://dumps.wikimedia.org/plwikinews/20110319> : Dump complete - 2011-03-19 14:49:07 fawiki <http://dumps.wikimedia.org/fawiki/20110319> : Dump complete - 2011-03-19 08:43:09 arwikisource<http://dumps.wikimedia.org/arwikisource/20110319> : Dump complete - 2011-03-19 08:41:44 vecwikisource<http://dumps.wikimedia.org/vecwikisource/20110319> : Dump complete - 2011-03-19 08:40:47 brwikisource<http://dumps.wikimedia.org/brwikisource/20110319> : Dump complete - 2011-03-19 05:27:47 fywiktionary<http://dumps.wikimedia.org/fywiktionary/20110319> : Dump complete - ... - 2011-03-19 00:33:09 plwikibooks<http://dumps.wikimedia.org/plwikibooks/20110319> : Dump complete - 2011-03-20 18:09:16 frwiki <http://dumps.wikimedia.org/frwiki/20110318> : Dump in progress - 2011-03-19 11:36:00 in-progress All pages with complete page edit history (.bz2) 2011-03-20 18:09:16: frwiki 53719 pages (0.488/sec), 7264000 revs (66.039/sec), 92.7% prefetched, ETA 2011-03-30 14:03:59 [max 63349860] - These dumps can be *very* large, uncompressing up to 20 times the archive download size. Suitable for archival and statistical use, most mirror sites won't want or need this. - pages-meta-history.xml.bz2 8.1 GB (written) - 2011-03-20 18:09:15 eswiki<http://dumps.wikimedia.org/eswiki/20110318> : Dump in progress - 2011-03-20 10:23:01 in-progress All pages with complete page edit history (.bz2) 2011-03-20 18:09:15: eswiki 5252 pages (0.188/sec), 1979000 revs (70.745/sec), 94.7% prefetched, ETA 2011-03-27 18:55:13 [max 44960402] - These dumps can be *very* large, uncompressing up to 20 times the archive download size. Suitable for archival and statistical use, most mirror sites won't want or need this. - pages-meta-history.xml.bz2 3.4 GB (written) - 2011-03-19 08:38:52 kbdwiki<http://dumps.wikimedia.org/kbdwiki/20110318> : Dump complete - ... - 2011-03-19 08:27:04 azwikisource<http://dumps.wikimedia.org/azwikisource/20110318> : Dump complete - 2011-03-19 00:13:10 itwikinews<http://dumps.wikimedia.org/itwikinews/20110318> : Dump complete

3 2

Problems with huwiki, frwiktionary and eswiki
by Andreas Meier 19 Mar '11

19 Mar '11

Hello, the following dumps are hanging during the extracted page abstracts for Yahoo part http://dumps.wikimedia.org/huwiki/20110316/ http://dumps.wikimedia.org/eswiki/20110318/ http://dumps.wikimedia.org/frwiktionary/20110316/ Best regards, Andreas

2 1

Jan history dumps (bz2 and 7z) now available
by Ariel T. Glenn 18 Mar '11

18 Mar '11

Well that, like many things about dumps, took longer than I would have liked but the January enwikipedia run is finally complete. Unless someone really really wants them (and then we might talk off list about it) I am not going to provide a single file for download of the history dumps; instead they are available in 15 pieces, both bzip2 and 7z compressed. The next run is under way; I expect it to take a while as well, as we are still working bugs out of the system. Please get your eyeballs on these, if you haven't picked them up already, and let me know of any issues with the contents. Thanks, Ariel

1 0

[xmldatadumps-l] Torrents
by White Cat 16 Mar '11

16 Mar '11

http://dumps.wikimedia.org/enwiki/20110115/ Hi, has anyone got plans to create individual torrents for "All pages with complete page edit history (.bz2)" ? I downloaded them and turns out I have several files that seem to be corrupted. I am unable to re-download them but feel the torrent would be able to fix the corrupted parts. All of the individual parts for the dumps except 1st,8th,9th,10th ones are complete. I need these dumps because I will analyse revisions in hopes of better identifying vandalism on the wikis through machine learning. I however need the database to process this soon as my assignment is due in about a month.

4 6

Upgrade of 7zip ?
by Nicolas Vervelle 15 Mar '11

15 Mar '11

Hi, It seems that the dumps are using an old version of 7zip (4.57 is visible in database dump progress <http://dumps.wikimedia.org/backup-index.html> page). Current stable version is 9.20 with apparently several optimizations done since 4.57. Should 7zip be upgraded for the dumps to see if it's going faster or taking less resources ? Nico

3 2

resumption of all dump jobs
by Ariel T. Glenn 09 Mar '11

09 Mar '11

After hearing a resounding silence from folks about whether the existing dumps are running well (perhas no news is good news?), I fixed up a small glitch which had caused the zh abstract dumps to fail under 1.17 and started up all jobs. Happy trails. Ariel

2 1

post-1.17 deployment restart of dumps
by Ariel T. Glenn 28 Feb '11

28 Feb '11

I have done a small amount of testing, the tests look good. Acccordingly I have started up one process to do dumps; please get your eyeballs on them and let me know thumbs up or down. I'd like to start up the rest of the processes by tomorrow at this time so if you can squeeze in some time to look at them sooner rather than later that would be awesome. Thanks! Ariel p.s. Yes this means I am done travelling for a while, thank goodness. I think I am sick of airplanes. And *very* sick of jet lag.

1 3

Problem about zhwiki
by xiang wang 24 Feb '11

24 Feb '11

Hello, As you know, Chinese contains two similar language: "Traditional Chinese" and "The simplified Chinese" , but it's hard to do translation between them correctly. I know Wiki can do this translation properly. I think why not release "Traditional Chinese" Dump and "The simplified Chinese" Dump, rather than together. This can save a lot of time for Chinese language researchers. Thanks. Just a serious advice!

1 0

upcoming 1.17 deployment and the xml dumps
by Ariel T. Glenn 09 Feb '11

09 Feb '11

A little bit before the scheduled deployment of the 1.17 branch on our production servers, I will be halting production of XML dumps. Deployment is set for Tuesday Feb 8 at 07:00 UTC, so a few hours before that I'll start shutting down processes. This is a precautionary measure; after the deployment and any hasty fixes that may be needed, I will be doing some testing to ensure that dumps are not impacted, before we restart them. Barring some bizarre problem, we should be back up and running within a day or two. Ariel

3 4

Problem with jawiki-dump
by Andreas Meier 01 Feb '11

01 Feb '11

Hello, there seems to be a problem with the current jawiki-dump. The size of the complete history dump is only 4.3 GB, but the size of the dump before was 19 GB. Another issue: Acccording to http://wikitech.wikimedia.org/view/Dumps#Worker_nodes there shoulde be 3 threads for the large dumps, but since a few days there are only 2 running threads. Best regards, Andreas -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone

2 2

← Newer
1
...
61
62
63
64
65
66
67
...
72
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Xmldatadumps-l