-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Dan Vanderkam wrote:
Hi, I noticed on http://download.wikimedia.org/enwiki/20070402/ that the ETA for the next history dump is May 19 and I have no reason to suspect this is wrong. Once the bz2 dump is finished, the 7zip dump will begin, and this is usually much faster. Last time, it took about ten days to complete after the bz2 dump had finished.
Is the 7zip dump generated from the bz2 dump or from the database?
- From the bz2 dump.
7zip is much slower to compress than bz2 (and the bz2 compression is additionally parallelized). The primary expense atm is pulling everything out of the db, particularly as the system realigns itself it currently has to grab a fresh copy of everything, and some things may be slower than they need to be, and some things will also take longer on older items than on newer, so the estimate may not be accurate.
- -- brion vibber (brion @ wikimedia.org)