On 14/03/11 19:52, Domas Mituzas wrote:
For now I have added a sleep() to the code to limit invalidations to 100 pages per second per job runner process.
Mass invalidations will create MJ issue in the long tail though... Need poolcounter ;-)
That's why Platonides wrote it for us. Now we just have to package it and deploy it.
There was a CPU spike after my change, but that was probably because 100 invalidations per second was still too fast, rather than a Michael Jackson type of problem.
On IRC, Daniel Friesen suggested pushing new HTML into the parser cache from the job queue instead of invalidating. Maybe after we move the parser cache to disk, that will be a good option.
-- Tim Starling