Hi,
don't know if this issue came up already - in case it did and has been
dismissed, I beg your pardon. In case it didn't...
I hereby propose, that pbzip2 (https://launchpad.net/pbzip2) is used
to compress the xml dumps instead of bzip2. Why? Because its sibling
(pbunzip2) has a bug bunzip2 hasn't. :-)
Strange? Read on.
A few hours ago, I filed a bug report for pbzip2 (see
https://bugs.launchpad.net/pbzip2/+bug/922804) together with some test
results done even some few hours before that.
The results indicate that:
bzip2 and pbzip2 are vice-versa compatible each one can create
archives, the other one can read. But if it is for uncomressing, only
pbzip2 compressed archives are good for pbunzip2.
I propose compressing the archives with pbzip2 for the following
reasons:
1) If your archiving machines are SMP systems this could lead to a
better usage of system ressources (i.e. faster compression).
2) Compression with pbzip2 is harmless for regular users of bunzip2,
so everything should run for these people as usual.
3) pbzip2-compressed archives can be uncompressed with pbunzip2 with a
speedup that scales nearly linearly with the number of CPUs in the
host.
So to sum up: It's a no loose and two win situation if you migrate to
pbzip2. And that just because pbunzip2 is slightly buggy. Isn't that
interesting? :-)
cheers,
--
Dipl.-Inf. Univ. Richard C. Jelinek
PetaMem GmbH - www.petamem.com Geschäftsführer: Richard Jelinek
Human Language Technology Experts Sitz der Gesellschaft: Fürth
69216618 Mind Units Registergericht: AG Fürth, HRB-9201
Dear Kevin,
It looks like <ftpmirror.your.org> may be down.
0) Time. I am not sure when it happened. Probably between 01:00 and 09:00
EDT.
1) TCP level. RSYNC, HTTP, and FTP all appear to be down.
(shell)$ rsync ftpmirror.your.org::
rsync: failed to connect to
ftpmirror.your.org(2001:4978:1:420::cc09:3752): Connection timed out
(110)
rsync: failed to connect to ftpmirror.your.org (204.9.55.82): Network is
unreachable (101)
rsync error: error in socket IO (code 10) at clientserver.c(128)
[Receiver=3.1.0]
(shell)$ wget http://ftpmirror.your.org/
--2014-05-28 13:07:32-- http://ftpmirror.your.org/
Resolving ftpmirror.your.org (ftpmirror.your.org)...
2001:4978:1:420::cc09:3752, 204.9.55.82
Connecting to ftpmirror.your.org
(ftpmirror.your.org)|2001:4978:1:420::cc09:3752|:80...
failed: Connection timed out.
Connecting to ftpmirror.your.org (ftpmirror.your.org)|204.9.55.82|:80...
failed: Network is unreachable.
(shell)$ ftp ftpmirror.your.org
ftp: connect: Connection timed out
ftp> quit
2) IP level. I can reach <ftpmirror.your.org> with PING but not with PING6.
(shell) ping -c 1 ftpmirror.your.org
PING ftpmirror.your.org (204.9.55.82) 56(84) bytes of data.
64 bytes from ftpmirror.your.org (204.9.55.82): icmp_req=1 ttl=52 time=33.9
ms
--- ftpmirror.your.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 33.958/33.958/33.958/0.000 ms
(shell)$ ping6 -c 1 ftpmirror.your.org
PING ftpmirror.your.org(ftpmirror.your.org) 56 data bytes
--- ftpmirror.your.org ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
3) Announcement. By the way, when you take a site down for maintenance, is
there anyplace I can look to see if there was an announcement?
Sincerely Yours,
Kent
Dear Kevin,
Early on 2014-May-01, just past midnight (EDT), a download using `wget'
broke off.
Since then, I cannot establish an HTTP connection to <
http://dumps.wikimedia.your.org>. I can establish an RSYNC connection to
get the list of modules:
(shell)$ rsync ftpmirror.your.org::
FreeBSD
FreeBSD-Archive
FreeBSD-CVS
wikimedia-dumps
wikimedia-images
wikimedia-imagedumps
centos
NetBSD
pkgsrc
kiwix
everything
Early this morning, I was able to list the contents of the `kiwix' module
and part of the contents of the `everything' module, but could not list the
contents of any of the `wikimedia-*' modules. This evening, I can not list
the contents of any of the modules:
(shell)$ rsync ftpmirror.your.org::wikimedia-dumps/
@ERROR: max connections (16) reached -- try again later
rsync error: error starting client-server protocol (code 5) at main.c(1534)
[Receiver=3.0.9]
Sincerely Yours,
Kent