Gregory Maxwell wrote:
It should never make things like watchlists slow, since it should be served 100% out of the front end caches. If it's causing slowness then there is a glitch which needs to be fixed and would likely have resulted in slowness no matter when it was run.
There was an overload on the frontend LVS load balancer when we enabled the fundraising banner at 01:30 UTC, which caused a cascading overload of the Florida squid servers. We disabled the banner almost immediately, but it took about an hour to isolate and fix the remaining problems, during which time the site was slow or down.
Frontend load was redistributed and the banner was phased back in between 04:50 and 05:15 UTC.
There are no fundamental capacity constraints which might warrant disabling or downsizing the banner. It's just that we encountered unexpected software problems as we hit new traffic records.
-- Tim Starling