I know Aaron has spent a lot of time on the job queue. But I have several observations and would like some feedback. The current workers apparently select jobs from the queue at random. A FIFO method would make far more sense. We have some jobs that can sit there in the queue for extended periods of time, while others added after that point may get completed in mere few minutes.
Second exposing job_timestamp via the API should also assist in identifying issues. whether or not some job is being ignored or the particular wiki is just extremely lagged.
I am monitoring a template modification to file links that occurred about hours ago, with a job queue between 350,000 and 500,000 items this delay seems excessive.