On Fri, Apr 25, 2014 at 11:13 AM, Fæ faewik@gmail.com wrote:
With no obvious immediate fix/work-around on the table from WMF ops, I have proposed to re-start my uploads for this project with an effective throttle by using 2 threads (this is a setting on the first screen of the GWToolset. In practice, having tried a run of a couple of hundred, this means that the tool is uploading 100MB sized images at a rate of 2 every 5 minutes. This seems to not be causing any issues.
The issue was not directly with the uploads; there is no thumbnail rendering happening on upload, so GWToolset adding lots of large TIFFs quickly would not cause problems in itself. The upload speed was problematic because that meant GWToolset saturated pages like Special:NewFiles, and when somebody looked at such pages, *that* triggered lots of thumbnail renderings of huge TIFF files at the same time. If GWToolset is slowed down and lots of miscellaneous files are uploaded between the TIFFs, those special pages won't be problematic, but something like a gallery or category of huge TIFF files could still be.