[Labs-admin] Today's Ops meeting & clinic followup 2017-06-26

[Andrew Bogott]

* There was some discssion during the Ops meeting about the Salt goal.  
In particular the goal says 'sunset' a lot of places and it's unclear if 
that means 'deprecate' or 'eliminate'.  We discussed briefly whether 
'eliminate' is possible within WMCS and I said that I think it is.

-- Here is where I found a tick crawling up my leg and freaked out 
slightly and missed a few things --

* There was a lot of talk about the issues with phab spam and 0-rating.  
It's a messy problem.  There's some discussion about turning off zero 
for the misc web, which would affect access to wikitech and horizon and 
some other of our services.

* I pointed people to the WMCS goals page but didn't enumerate them.  
Mark asked, 'anything that will affect the rest of us'? and I said "Not 
really."


Here are the things that happened during clinic last week:

-Removed a bunch of old files from labmon1001

-Cleared a disk space warning on install1002

-Dealt with a brief proxy outage that I also caused

-Rebooted all labvirts and openstack boxes for stack clash

-Handled a few report requests related to downtime and stopped services 
during and after the reboots

-Brief instance creation outage on Thursday, fixed by restarting 
rabbitmq-server

-There was some (unresolved) concern that normal phab users can't 
comment on WMCS blog posts.

-Neutron is filling up the labtestnet2001 drive over and over, it only 
takes an hour or so to fill.  For now, Neutron is turned off to stop 
this happening.