On Mon, Feb 16, 2009 at 11:02 AM, Thomas Dalton thomas.dalton@gmail.com wrote:
Alternatively, the number of jobs processed per request could be made a function of the length of the backlog (in terms of time) - the longer the backlog is, the faster we process jobs. Then if the job queue get to being months behind we would all notice it because everything would start running really slowly.
Jobs are not processed on requests. They're processed by a cron job. You can't just automatically run them at a crazy rate, because that will cause slave lag and other bad stuff. If too many are accumulating, it's probably due to a programming error that needs to be found and fixed by human inspection. (Tim just made several commits fixing things that were spewing out too many jobs.)
(Obviously, the length of the job queue needs to be added to whatever diagnostic screen the devs first check when the site slows down, otherwise it won't help much.)
#wikimedia-tech has enough people that regular warnings posted there would probably get noticed.