[Foundation-l] [Xmldatadumps-l] Wikipedia dumps downloader

[Milos Rancic]

On 06/27/2011 01:07 PM, emijrp wrote:
> Yes, a distributed project would be probably the best solution, but it is
> not easy to develop, unless you use a library like bittorrent, or similar
> and you have many peers. Althought most of the people don't seed the files
> long time, so sometimes is better to depend on a few committed persons than
> a big but ephemeral crowd.

If you create a way to sync images partially (let's say, the base
letters inside of the directory structure), a couple of dozens of
Wikimedians could create a couple of dozens of backups. I could host on
my personal computer ~500GB, which approximates one letter.