On Thu, Jun 25, 2009 at 8:14 PM, Domas
Mituzas<midom.lists(a)gmail.com> wrote:
The problem is quite simple, lots of people
(like, million pageviews
on an article in an hour) caused a cache stampede (all pageviews
between invalidation and re-rendering needed parsing), and as MJ
article is quite cite-heavy (and cite problems were outlined in
http://article.gmane.org/gmane.science.linguistics.wikipedia.technical/41547
;) the reparsing was very very painful on our application cluster -
all apache children eventually ended up doing lots of parsing work and
consuming connection slots to pretty much everything :)
So if two page views are trying to view the same uncached page at the
same time with the same settings, the later ones should all block on
the first one's reparsing instead of doing it themselves. It should
provide faster service for big articles too, even ignoring load, since
the earlier parse will be done before you could finish yours anyway.
It's quite a complex feature. If you have a server that deadlocks or
is otherwise extremely slow, then it will block rendering for all
other attempts, meaning that the article can not be viewed at all.
That scenario could even lead to site-wide downtime, since threads
waiting for the locks could consume all available apache threads, or
all available DB connections.
It's a reasonable idea, but implementing it would require a careful
design, and possibly some other concepts like per-article thread count
limits.
-- Tim Starling