From a user perspective, it would be neat if we could
set something
like "high / medium / low" priority at the time of creating
the upload
request. Some GWT users might be trying to process their image
collection in time for a planned edit-a-thon or similar, so taking a
week to upload might be a problem (it is taking me this long to get
through the NYPL maps uploads).
This would give operations more choices on how to handle medium/low
priority requests, as it would be fair to intermittently pause these,
or make them as slow as you like, during peak demand times. In fact it
would make sense if a queue of "low" priority were limited to sharing
the one processing thread.
All of this really works if GWT can give the user some feedback on
progress. At the moment the only way I know of doing this is to run an
series of API queries checking whether everything on my list of images
exists yet or not, itself probably creating more stress on the
servers.
Fae
On 29/04/2014, Aaron Schulz <aschulz(a)wikimedia.org> wrote:
If they are not "delayed" then yes, it
won't do much to limit the files/per
job. Setting $wgJobBackoffThrottling might be useful here.
--
faewik(a)gmail.com
https://commons.wikimedia.org/wiki/User:Fae