On Wed, Jul 23, 2003 at 06:13:17PM -0700, Daniel Mayer wrote:

Tomasz Wegrzanowski wrote:

...

We have plenty of bandwidth on wikipedia.org,

For now at least. But the various Wikimedia projects are together creating new pages at an exponential rate that are in turn being viewed by an exponentially increasing number of readers.

We have been very fortunate to have a bandwidth sugar daddy (hi Jimbo!) so far but we should not assume that Bomis will be able to foot the bill when our popularity (measured in hits/bandwidth consumption) becomes comparable to websites like Slashdot.org or even Yahoo.com.

At some point we are going to have to economize our bandwidth usage. BitTorrent is one of those possibilities (mirrors are another) and IMO it is better to start thinking about this now rather than when bandwidth gets more limited. I have, however, no opinion on this particular technology though.

While usage of bandwidth by Wikipedia may grow exponentially, cost of gigabyte transfered keep falling exponentially, so you shouldn't care that much. (http://xp.c2.com/YouArentGonnaNeedIt.html)

Anyway using BitTorrent makes completely no sense in this context - it's for big static stuff that masses want, like RedHat .isos, porn or DVD rips. Neither with articles (which are way too small) nor dumps (which aren't static and too few people want them) we would benefit from using BitTorrent.

If we change our database dumps every day, we should be providing some kind of .diffs instead. On late phase 1 rsync was used and it reduced bandwidth on both client and server by more than an order of magnitude (albeit at high CPU cost). We should rather go in this direction.