[Foundation-l] Wikistats is back

[David Gerard]

2008/12/25 Erik Zachte <erikzachte at infodisiac.com>:

> Hi Brian, Brion once explained to me that the post processing of the dump is
> the main bottleneck.
> Compressing articles with tens of thousands of revisions is a major resource
> drain.
> Right now every dump is even compressed twice, into bzip2 (for wider
> platform compatibility) and 7zip format (for 20 times smaller downloads).
> This may no longer be needed as 7zip presumably gained better support on
> major platforms over the years.
> Apart from that the job could gain from parallelization and better error
> recovery.


7zip is readily available as free software for Unixlike platforms,
though it's pretty much never installed by default.


- d.