Hi all,
There was no ops meeting this week, here are the things I looked at during clinic duty last week:
* Debugging and fixing PAWS ** https://phabricator.wikimedia.org/T183142#3862801 ** Pending: Removing the blacklist for overlayfs for paws cluster.
* Wiki-replica analytics high replag reports on mailing list ** Created https://phabricator.wikimedia.org/T183983 following the discussion in today's meeting. ** Need to follow up with DBAs
* Followed up on tools and misc high NFS storage utilization ( https://phabricator.wikimedia.org/T183920) and made subtasks for the relevant project admins.
* There were couple reports of the bastion being slow, but both times got resolved by folks other than me before I even got to look at the pings * Resolved a bunch of toolforge membership requests * Extended icinga downtime for labtestcontrol2003 * Found puppet staleness in labstore1004 & 5 - Chase has already looked at this and resolved.
* I responded to production pages for pdfrender late night on Christmas Eve and restarted services on scb1001-4 to resolve since I was the only one around.
* The cloud list was active this week! ** People helping yay! *** https://lists.wikimedia.org/pipermail/cloud/2017-December/000137.html *** https://lists.wikimedia.org/pipermail/cloud/2017-December/000149.html ** Bryan is awesome at responding to user questions - https://lists.wikimedia.org/pipermail/cloud/2017-December/000136.html ** Concerning long thread on user tables https://lists.wikimedia.org/pipermail/cloud/2017-December/000112.html
That's all I think!
Best,