On Fri, Apr 25, 2014 at 11:13 AM, Fæ <faewik(a)gmail.com> wrote:
With no obvious immediate fix/work-around on the table
from WMF ops, I
have proposed to re-start my uploads for this project with an
effective throttle by using 2 threads (this is a setting on the first
screen of the GWToolset. In practice, having tried a run of a couple
of hundred, this means that the tool is uploading 100MB sized images
at a rate of 2 every 5 minutes. This seems to not be causing any
issues.
The issue was not directly with the uploads; there is no thumbnail
rendering happening on upload, so GWToolset adding lots of large TIFFs
quickly would not cause problems in itself. The upload speed was
problematic because that meant GWToolset saturated pages like
Special:NewFiles, and when somebody looked at such pages, *that* triggered
lots of thumbnail renderings of huge TIFF files at the same time. If
GWToolset is slowed down and lots of miscellaneous files are uploaded
between the TIFFs, those special pages won't be problematic, but something
like a gallery or category of huge TIFF files could still be.