hey Gilles,
On Tue, Sep 2, 2014 at 5:25 PM, Gilles Dubuc gilles@wikimedia.org wrote:
Hi Filippo,
I've updated the changeset to support both cURL hits and rendering inside the job. Sadly it seems like cURL hits also need to be a job, because I tried sending a fire-and-forget HEAD request (with a Connection: Close header) and that wasn't enough to trigger the thumbnail generation. Thus the cURL request has to wait for completion, and a result is inside a job in order to avoid the upload script hanging while the thumbs are being generated. The upside is that the job that sends the cURL requests could be rate-limited.
good to know! FWIW I think we are better off with the job anyways
BTW what sizes are you planning to start out with?
Mostly the sizes used by Media Viewer and probably the most common
thumbnail sizes found in articles as well. I'm open to suggestions regarding how many different sizes should be pregenerated and which ones would be reasonable to pick. I guess maybe it's best to start with a minimal set and then expand it later if the resource usage is reasonable?
+1 to the minimal set, perhaps the top 2/3 most common resolutions for devices using MV? let us know when you plan to deploy it so we can keep an eye on swift.
thanks! filippo