[Foundation-l] State of technology: 2007

[Domas Mituzas]

Hello,

> Hi.  Fabulous colleague-shareholder-report you have there; but could
> you fix the transition speed on your powerpoint slides?  and the font
> in the footer...

Transition speed? Powerpoint slides? That is a workbook, not a  
presentation :) people read it as a pdf book! :)

> Is this a general trend -- remote hands not being available during
> critical moments -- or a chicken-and-egg issue with other elements
> (such as Rob being around)?

It gets complicated with all operations folks being in Europe -  
hardware provisioning, reinstalls, etc - is usually managed not from  
U.S. That limits the time window for various work. Datacenter ops are  
more fixed in time than anything else - and when we needed things to  
do ASAP, people were in schools/jobs/etc.

Now with Rob around we can be pretty sure that any critical issue can  
be dealt with swiftly, and non critical jobs still in reasonable time.

> This is cool; I had no idea.  Is there a longer description of how  
> it works?

Probably Mark could tell much more, but generally equal providers  
like to exchange their traffic for free - they hop on to traffic  
exchanges (think of a huge switch, well, in real world it is bunch of  
big switches :), and look for peering partners. Usually it is a bit  
of trouble for content providers to get peering, but Google has it  
with nearly every major provider. For smaller people like us we have  
to be really cool. Mark did lots of social work to present us as cool  
in networking world, and we are allowed to use some free resources.

The press release about such activities in Amsterdam was at:
http://wikimediafoundation.org/wiki/AMS-IX

> Any offers of support from multiple really big donated hosting places?

The big problem is that it has to have hardware - lots of it - to  
support our cached data set, and if you try to disperse it over  
multiple datacenters, really complex problems start like how to  
balance the requests so they go to datacenter which would have that  
data. Stuff like 'lets have french go here, germans there' adds lots  
of administrative work - from maintaining all the platform, to  
actually troubleshooting.

Generally, more datacenters are there, more probably we'll miss  
problems. This was especially seen by moving Asian languages to Asia  
- by having platforms we manage less than Tampa we'd eventually end  
up with them working slower, more errors, etc. What was interesting -  
no Asians would come and tell that to us - it seems that everyone is  
used there to bad international sites. Once we reduced the complexity  
and did just what was nice, easy to manage and efficient - we ended  
up having people in those countries tell 'yay it is very fast, faster  
than local sites'.

So, if someone would come up with big donated place, that would bring  
as much caching hardware as we have now in Amsterdam, or in Tampa -  
probably it could be possible to consider. Still, for many of these  
places to host us would be as expensive as cover our costs. And by  
using more sites, our costs increase.

> Are there recent stats on the # of reusers, sites, contributors;
> mediawiki extension variants / repositories outside the main tree?

Well, quite a few people commit extensions to our repositories:
http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/

The community outside our #mediawiki cabal has not formed, though  
there're various extensions floating around, that are more suitable  
for other sites, than us. And there're lots of small wikipedias or  
wikimedias all around the world (I run one at job, management never  
called it 'wiki' - it sounded as old not very nice software, or  
mediawiki).

> Ditto for stats on # and quality of patches from colleagues from other
> shops.  Is there a wall of huge-site-heroes for those who release
> their patches?

I mean more the knowledge sharing with people who run big non- 
mediawiki sites. Like, folks from Yahoo helped with APC. Google have  
nice patches for MySQL. SixApart help with memcached. Even though  
they don't directly fix our bugs, their engineers are willing to  
communicate, discuss the operations, and sometimes improve software  
we use. And in the end they even can buy you a drink. :)

Best regards,
-- 
Domas Mituzas -- http://dammit.lt/ -- [[user:midom]]