Jeremy Dunck wrote:
According to the hardware orders page[1], a major bottleneck is page rendering.
Does any phase of this stand out? DB fetch? Text parse? Request/Response bandwidth?
I assume profiling on this has been done quite a lot...
[1] http://meta.wikimedia.org/wiki/Hardware_ordered_August_30%2C_2005
See http://meta.wikimedia.org/wiki/Profiling/20050822. I don't think I'd call any of it a bottleneck, optimisation aimed at reducing average page view time or CPU load is hard work. I suspect there are hotspots, but not in our code, in the PHP VM. I think the best way to reduce page view time at this stage would be to optimise that, or to produce a JIT compiler.
It's much easier to produce user-visible speed improvements by concentrating on relatively rare but slow operations. These often account for a similar amount of the average request time as do the parser "hotspots". And they've often been neglected by past optimisation work.
For example, I notice on the profiling results I posted above, the most expensive database query is from oaiUpdatePage(). That's not a page view query at all, it's a save query. It's a relatively new feature, and it probably hasn't been examined in the optimisation context. It's apparently adding 1750ms to save times, which is a huge amount, easily user-visible.
Another similar possibility is to work on optimisation of pathological page views, rather than typical page views. It's easy to construct a page that takes more than 30s to render. I recently did some work on pathological diffs and pathological image scaling, and in both cases acheived a speedup on the order of 100x for the test case.
Optimisations such as these do reduce our costs, but they also improve user experience -- disproportionately so, compared to a 5% reduction in time for an already subjectively fast operation. They also improve robustness -- something which adding more hardware cannot do.
-- Tim Starling