I forgot some links to show: 
1- Scoring requests: https://grafana.wikimedia.org/dashboard/db/ores?panelId=1&fullscreen&from=1476641447816&to=1476652262851
2- Response time: https://grafana.wikimedia.org/dashboard/db/ores?panelId=15&fullscreen&from=1476641447816&to=1476652262851
3- uwsgi busy workers: https://grafana.wikimedia.org/dashboard/db/ores?panelId=13&fullscreen&from=1476641447816&to=1476652262851
4- icinga screams: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-ai/20161016.txt

Best

On Mon, Oct 17, 2016 at 12:39 AM Amir Ladsgroup <ladsgroup@gmail.com> wrote:
Hey,
It seems there is some sort of back pressure on the ORES service right now causing to send out timeout and overload errors which made icinga scream at #wikimedia-ai several times today. If you're running the requests, please slow down a little.

We Increased the capacity for now [1] (Thanks to Andrew Bogott). That brought back everything to the normal state. Sorry for any inconvenience.

[1] https://gerrit.wikimedia.org/r/#/c/316271/
Best