On Thu, Jul 10, 2014 at 3:08 PM, Gergo Tisza <gtisza@wikimedia.org> wrote:

Hi all,

with bug 65691 fixed (the last patch was deployed today), now might be a good time to test large TIFF uploads again (the patch is limited to TIFF files for now). I was thinking of the following schedule:

- wait until Monday (no breaking the site on the weekends)

- launch an upload with large TIFF files (preferably the same one that caused issues earlier, ie. Fae's NYPL maps project)

- make sure that the images are initially not categorized to avoid someone triggering 200 new thumbnail requests in parallel (GWToolset could add an emtpy template instead, and that template can be replaced with the category later).

- initially use the minimum speed allowed by GWToolset (a single thread), to make sure Special:NewFiles and co. will also not be the source of many concurrent requests.

- after a bunch of images have been uploaded, generate a gallery with 10 thumbnails and monitor imagescaler load and Swift traffic in the process. Repeat with 20, 50 etc until we are satisfied that the scalers are resilient to many concurrent requests for large files.

- if all works out, the upload project can continue with normal speed (20 threads or whatever), and we can also relax throttle limits on GWToolset a bit.

Does this sound reasonable? Fae, are you interested in doing this?

For the record, this is on hold because no one seems interested in doing mass TIFF uploads at the moment. Whenever someone plans doing that, ping me or anyone from the Multimedia team and we can get back to this.