I don't think I ever sent these out to the group. :/ == 2017-11-28 - 2017-12-04 == Tues: * Superyetkin irc about error.log output * Superyetkin irc about user dbs * Created massmessage project Wed: * Paladox spotted a Puppet problem caused by refactoring; Andrew fixed it * Talked to matanya about his collaboration project request * Created collaboration project * Talked to matanya about ssh issues connecting to tools-login (looks like routing) Thur: * Fixed a bug in stashbot that anomie noticed * Superyetkin and https://phabricator.wikimedia.org/P6409 * Krinkle and https://phabricator.wikimedia.org/T181742 * Quota reduction -- https://phabricator.wikimedia.org/T177299 * Pinged for a 2FA removal -- https://phabricator.wikimedia.org/T181475 * Verified and closed https://phabricator.wikimedia.org/T176043 * Made a maintenance script to attach LDAP accounts to wikitech for https://phabricator.wikimedia.org/T180813 * Attached Flominator and set email to match SUL account * Triaged some random bugs in our backlog * Made some traffic report dumps: ** https://phabricator.wikimedia.org/P6413 ** https://phabricator.wikimedia.org/P6414 Fri: * Phab triage * Worked on https://phabricator.wikimedia.org/T171417 Sun: * Crontab encoding -- https://phabricator.wikimedia.org/T181948 == 2017-12-05 - 2017-12-11 == Tues-Sun: * offsite + kubecon; didn't keep notes or do much Mon: * looking at tools-static cndjs disk space -- https://phabricator.wikimedia.org/T182604 TechOps: * Code stewardship reviews (previously called "Sunsetting") is now live: https://www.mediawiki.org/wiki/Code_stewardship_reviews ** Faidon has a few things he is thinking about proposing for review (e.g. irc feed which would be a thing that Toolforge users care about) * Singapore DC racking done (except for serial console which was DOA hardware) ** Planning for full turn up in Q3 * k8s 1.7.10 upgrade in progress by Alex * s8 is "done" but rolling out in Q3 (2018-01-09) * All prod puppetmasters on Puppet4; Keith working on clients (small bug working on now); PuppetDB will not be upgraded this quarter ** Puppet4 syntax should be ok to use in ops/puppet now * Ganeti servers are having some issues <https://phabricator.wikimedia.org/T181121> * netbox should have all the racktables data; test it if you'd like * Q3 draft goals -- https://etherpad.wikimedia.org/p/TechOps-goals-FQ3-FY1718 ** Asia DC ** Varnish 5 for all clusters ** Continue DB backups program ** Multi-DC: deploy Etcd/conftool for MediaWiki ** esams cleanup (would span into Q4) ** Q3 procurement/refresh/expansion *** Try to get all racked hardware into service ** Puppet4 completion (PuppetDB) (and look at Puppet5? needs work to flesh out) ** Streamlined Services: 1 prod service on k8s ** infrastructure monitoring (needs details) Bryan -- Bryan Davis Wikimedia Foundation &lt;bd808(a)wikimedia.org&gt; [[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA irc: bd808 v:415.839.6885 x6855