[WikiEN-l] Thankyou to all the developers.

I said on the village pump that I thought we are doing a great job, but I don't think any of the developers I was aiming that comment at read it. Let me take this opportunity to thank everyone and recognise what we have acheived. We've been working in adverse circumstances. During peak times, the site has been extremely heavily loaded and unstable. Any slight error in misconfiguration, or inaction at a particular time when action should have been taken, causes the site to crash. I've lost count of how many problems we've identified and fixed just over the last few weeks. Many users say "the developers are doing a great job", or "we all know that the developers are very busy", but the fact is that 99% of users don't have a clue what we are doing. They don't know what our achievements have been and they don't know the challenges we face. Gerard's comments are certainly refreshing in this regard, but I think I can add to them. Assume I am speaking on behalf of the users, since I'm sure every user would agree with me if they only knew who to thank. Big thanks go to JamesDay, who almost single-handedly administers 8 database servers, a task requiring constant monitoring and work on the order of hours per day. James's advice to the MediaWiki developers and other system administrators is invaluable. Also on the topic of database administration, Kate's servmon and WikiServices bots which have kept the site running when otherwise it would have been choked with long-running queries. Thanks to Med and Submarine for their work in network and hardware administration of the Paris squids. Well done Mark and Kate for getting PowerDNS up and running and thus getting the Paris squids into service. Domas's setproctitle() patch is amazing and we all know it. Of course his other system administration and development work is greatly appreciated. Thanks to Brion, JeLuF and Hashar for their tireless and usually unrecognised work in fixing MediaWiki bugs. Thanks to Kate for setting up Pen and Perlbal. This is the third time I'm thanking Kate and that's no coincidence - if she left us we'd be left with a dozen pieces of software that no-one else knows how to use. I know you're all stressed, all we seem to get is complaints despite what we've acheived. I decided after Caroline Ewen's post on wikipedia-l that I can't afford to answer every single question asking "why is the site slow" or every report of "I'm getting backtrace errors!" The fact is that the site has grown to such a size that every time something goes wrong, we can expect a flood of complaints and queries. My advice would be to answer only some of them, and let alert users distribute the information to everyone who asks. Or just ignore them -- remember your time is valuable. Think of what you could have achieved in the time it took you to put a single user out of their ignorance. When the public forums are too noisy with uninformed speculation, let's organise what we need to make the site better in less visible, more constructive forums, and work with the Board to make it happen. -- Tim Starling

Just to take this a bit further. I thought I'd compare Wikipedia with one of the "well-run sites" that we are supposed to be competing with. Google is a good direct comparison, because of its dynamic content, with cachable frequent queries. Looking at the difference in traffic between Google and Wikipedia on Alexa shows that: * Wikipedia has 300 page views per million * Google has 16,000 page views per million Thus, Google serves roughly 53 times the number of page views compared to Wikipedia. However, * Wikipedia currently has 39 servers * Google has an estimated 50,000 - 100,000 servers in its worldwide farm of clusters Thus, Google has roughly 1250 - 2500 times as many servers as Wikipedia [Source: http://www.tnl.net/blog/entry/How_many_Google_machines for an estimate for April last year, and allowing for more recent expenditure] Thus, we might regard Wikipedia as being roughly 24 to 48 times more "efficient" in its use of hardware than Google. Given that Google has spent over $250M on hardware, to obtain reasonable parity for our developers to be expected to compete with Google at our current traffic we should have around 1000 - 2000 high-performance servers, at a cost of several million US$. So, a reasonable answer to critics seems to be: * the developers are already doing very well indeed coping with the combination of extremely high demand and very limited resources * they already know there are big growth and capacity problems, and are working hard on scalability and reliability * send money, rather than complaining -- Neil