Tim Starling wrote:
It's probably better to use rsync, that's what we usually use for image backups. It's quite robust, and efficient if used properly. The only trouble with it is the server load -- it would quickly overload the backend if we made it available for public use. Maybe we could set up an rsyncd instance limited by client IP.
Note that rsync takes too long to build the file list for the full set of files; you have to break it up by smaller directories to get a reliable transfer.
-- brion vibber (brion @ wikimedia.org)