On Thu, May 20, 2010 at 11:30 AM, William Pietri william@scissor.com wrote:
Good question. I should say that I have no inside knowledge on this project, and am speaking purely as a random Wikipedian who does web stuff for a living. That's just my educated guess, both on ratios and clicks.
Personally, I'd love to see more data, both as a driver for interface decisions and so that we can measure the results. But realistically, I can't hold Wikipedia to the same standards I hold my other clients.
Wikipedia was run on a shoestring for years, with the main goal being survival under massive load. It's easy enough for a low-traffic startup to add decent instrumentation, but adding it to an existing system that's been heavily optimized for high volume and incredibly low cost? Getting good data there will likely be a lot of work.
If you don't know something then don't say something. We're not benefited from the blah blah.
Most of the super-optimization of the site is related to ensuring that users are cache hits that never make it to the backend.
Proper testing is either sampled (using JS tracking code handed out to a subset of users), or in the case of things like search is simply measured purely from the backend since the front end doesn't satisfy these things.
Both kinds of measurements have been done before on English Wikipedia.