This might slow things down in the normal case and tie up more scalar workers in the higher load case when outages previously happened. I'd lean towards using the priority based pool countering or something similar for now (see Tisza's comment in the other ops thread).

Perhaps the CURLOPT_MAX_RECV_SPEED_LARGE limit could just be applied to files in the "expensive" group if this flag were to be used.

On Fri, May 9, 2014 at 3:24 AM, Gilles Dubuc <gilles@wikimedia.org> wrote:

It seems like the CURLOPT_MAX_RECV_SPEED_LARGE option [1] can do the job. I guess now the questions is: what limit should we set?

During the image scaler outage Faidon mentioned that the swift bandwidth was maxing out. Do you remember what the cap was?

[1] http://www.php.net/manual/en/function.curl-setopt.php

_______________________________________________
Ops mailing list
Ops@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ops

--
-Aaron S