[Labs-admin] On-call summary for week of 2017-08-21

[Andrew Bogott]

Ops meeting

- Salt deprecation:  no mention of labs but good progress on deployment 
tooling.  OCG is an outlier here but that's mostly up to the releng people

- Preparing for puppet 4: Giuseppe is chasing down catalog differences 
between 3 and 4 for production hosts (summary at 
https://puppet-compiler.wmflabs.org/compiler02/7622/index-future.html) 
Probably we should look at our servers that show up on that list. Of 
course we don't really have a way to do this for labs instances...

- Moritz is going to move Ganglia behind ldap auth.  I'll notice this, I 
don't know if anyone else looks at ganglia anymore.


Nothing very interesting happened for clinic, but here are some things I 
did:

- Responded to yet another rabbitmq/nodepool storm.  Restarted rabbitmq, 
turned off nodepool, waited a few minutes, turned everything back on, 
all fixed.

- Updated wikitech-static (in response to icinga alert)

- Noted that the *.wmflabs.org cert is expiring and handed off to Robh 
to renew

- Cleaned up some log files on silver to silence an icina warning. 
Haven't bothered to fix this for real since silver is not long for this 
world.

- Misc. irc support